These notes assume instructions are followed and that there is an initial node created.
Open internode ports
Using OCI networking tools, open the following:
| Stateless | Source | IP Protocol | Source Port Range | Destination Port Range | Type and Code | Allows | Description |
|---|---|---|---|---|---|---|---|
| No | 10.0.0.0/16 | TCP | All | 25000 | — | TCP traffic for ports: 25000 (MicroK8s) | |
| No | 10.0.0.0/16 | TCP | All | 10250–10260 | — | TCP traffic for ports: 10250–10260 (MicroK8s) | |
| No | 10.0.0.0/16 | TCP | All | 12379 | — | TCP traffic for ports: 12379 (MicroK8s) | |
| No | 10.0.0.0/16 | TCP | All | 19001 | — | TCP traffic for ports: 19001 (MicroK8s) | |
| No | 10.0.0.0/16 | UDP | All | 4789 | — | UDP traffic for ports: 4789 (MicroK8s) | |
| No | 10.0.0.0/16 | TCP | All | 179 | — | TCP traffic for ports: 179 (BGP) (MicroK8s) | |
| No | 10.0.0.0/16 | IP-in-IP | — | — | — | IP-in-IP traffic (MicroK8s) | |
| No | 10.0.0.0/16 | ICMP | All | All | — | ICMP traffic for: All (MicroK8s) |
For each node
Get a VM on the same network as your master node and follow instructions from https://kambur.ie/tips/k8s/microk8sinoci/ optionally skipping External Access section. This in fact will create dependency on the original node being up but we will come back to it.
Run the following to open ports.
# Allow MicroK8s dqlite
sudo iptables -I INPUT 5 -s 10.0.0.0/16 -p tcp --dport 19001 -j ACCEPT -m comment --comment "MicroK8s dqlite"
# Allow MicroK8s cluster agent
sudo iptables -I INPUT 5 -s 10.0.0.0/16 -p tcp --dport 25000 -j ACCEPT -m comment --comment "MicroK8s cluster agent"
# Allow MicroK8s apiserver HA communication
sudo iptables -I INPUT 5 -s 10.0.0.0/16 -p tcp --dport 12379 -j ACCEPT -m comment --comment "MicroK8s apiserver-ha (etcd-like)"
# Allow kubelet and related control-plane traffic
sudo iptables -I INPUT 5 -s 10.0.0.0/16 -p tcp --match multiport --dports 10250:10260 -j ACCEPT -m comment --comment "MicroK8s kubelet and node comms"
Add nodes as here: https://microk8s.io/docs/clustering
Note that if using AMD Free Tier machines, probably best to use them just as workers.
To test:
kubectl run my-shell --rm -i --tty --image nicolaka/netshoot --overrides='{"spec":{"nodeName":"worker-1"}}' -- bash
Using this, you can force K8s to schedule the my-shell pod on newly added worker. Check connectivity to outside world and to the cluster.