Kubernetes Raspberry pi(no Microk8s, k3s)
So, let’s get to the truth. This is not the first article you are coming across for deploying your “homelab” on a affordable yet powerful tool for understanding kubernetes stack. I followed many articles but some parts were missing in every article I read. So, these steps should work for you to learn how to setup a kubernetes deployment in 2 or more raspberry pi 4B.
Few key points good to know:
- runc(this is the actual container implemented using gRPC and protocol buffers)
- Docker/CRI(to control runc)
- Having structural knowledge of kubernetes is always good.
Note: I will not recommend anything below raspberry pi 4B.
Okay, you will need the following to start with:
- 2 raspberry pi 4B (4gb)
- 2 microSD cards(anything greater than 8 gb is good). But why bother when you can get the 32gb at nearly the same price.
- Two chargers 5V~3A(usb C type). Don’t want to include the messy cables — use PoE.
2. Download cni binaries for arm.
3. Download ubuntu 20.04 images
5. Heat sinks(optional), while running the stack check if /sys/class/thermal/thermal_zone*/temp is below 80°C. If yes, you do not need this.
First things first, download Balena Etcher or any of your favourite tools to create bootable SD cards using ubuntu 20.04 images.
SSH into the machines and make a new user for both master and worker node.
update and upgrade your distro:
sudo apt update && sudo apt upgrade
For initial setup of your ubuntu images on master and nodes, Adnan Strujic shares a great guide.
Quick check: Till now you have booted and are able to ssh into both master and worker with “ssh master/worker”
Make amendments in /boot/firmware/cmdline (This is the first file that is run when firmware is looking what to do next.) You can unleash the dragon(linux) by amending this file(not for newbies). If you are running linux see the first line of “dmesg” to confirm if it is the first file that boots.
Now your machine is ready to install kubernetes.
Next, download all binaries:
sudo apt update && sudo apt install -y kubelet kubeadm kubectl
Setup host modules:
br_netfilter’ | sudo tee -a /etc/modules-load.d/containerd.conf
sudo modprobe overlay
sudo modprobe br_netfilter
echo ‘net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1’ | sudo tee -a /etc/sysctl.d/99-kubernetes-cri.conf
sudo sysctl — system
Note: Mind the newline character for each entry using echo.
Install containerd with docker(as they were the initial developers of containerd):
# Install containerd
## Set up the repository
### Install packages to allow apt to use a repository over HTTPS
sudo apt-get install \
software-properties-common### Add Docker’s official GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -### Add Docker apt repository.
sudo add-apt-repository \
"deb https://download.docker.com/linux/ubuntu \
stable"## Install containerd
sudo apt-get update
sudo apt-get install containerd.io# Configure containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml# Restart containerd
sudo systemctl restart containerd
Unfortunately, I could not find much on internet for arm64 for CRI-O.
Note: you may find that systemd is not loaded while initializing containerd. Follow this for solution: https://github.com/containerd/containerd/issues/4900#issuecomment-755937136
Create a socket for connecting containerd and CRI(hand of god of kubernetes)
sudo nano /etc/crictl.yaml
We need to tell kubeadm about the cgroup driver and socket.
sudo nano /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
Environment=”KUBELET_EXTRA_ARGS= — cgroup-driver=systemd — container-runtime=remote — runtime-request-timeout=15m — container-runtime-endpoint=unix:///run/containerd/containerd.sock”
reload setting without restart:
sudo systemctl daemon-reload
sudo systemctl restart kubelet.service
But how will the communication take place between the pods, services and other nodes’ components? Weavenet to our rescue:
kubectl apply -f “https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d ‘\n’)
See the magic happen by initializing kubeadm on master node:
sudo kubeadm config images pull
sudo kubeadm init — cri-socket /run/containerd/containerd.sock
Upon completion of 2nd command, you should have the token from master now. Use the token to connect by running below command(##THIS IS A RANDOM TOKEN AND HASH## USE THE OUTPUT THAT YOU HAVE OBTAINED##):
sudo kubeadm join 192.168.184.186:6443 — token o92btx.6lkh97nb3m7f7 — discovery-token-ca-cert-hash sha256:1dee61cf705803f6284573a987e82654e639b3948b6cab7ffdb793b762
Now, run command “kubectl get nodes” to check if the nodes are in ready state.
Your kubernetes-system stack is ready now. Enjoy!!