Having multiple homegrown k8s clusters at former work… I missed them (sad individual I am) so I got mine

Steps to follow to get microk8s running on Ubuntu 24 and arm with great help of ChatGPT and ClaudeAI

If you follow instructions on how to install microk8s on Ampere VM running Ubuntu 24 in OCI, you’ll discover that while install succeeds, pods can’t communicate.

It would appear there are 2 separate issues that prevent it from working:

1. microk8s requires legacy iptables
2. some default routes issue that appears to be caused by default install in OCI.

Workaround legacy IP tables issue

Before installing microk8s:

sudo apt-get update
sudo apt-get install -y iptables arptables ebtables

And then, alternatives switch:

sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
sudo update-alternatives --set arptables /usr/sbin/arptables-legacy
sudo update-alternatives --set ebtables /usr/sbin/ebtables-legacy

To verify:

sudo update-alternatives --display iptables
sudo update-alternatives --display ip6tables
sudo update-alternatives --display arptables
sudo update-alternatives --display ebtables

all should be displaying that legacy binaries are in use.

Install microk8s

Follow steps 1&2 from instructions.

Update iptables

It would appear that the following iptables rule is on the wrong place… moving it fixes connectivity issues.

sudo iptables -D FORWARD -j REJECT --reject-with icmp-host-prohibited
sudo iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited

This needs be done each time after restart. These notes will be updated to reflect more permanent solution.