Kubernetes Multi-Node Cluster on Raspberry Pi 4

Share on:

Overview

In this article we'll setup a 3 nodes Kubernetes Cluster running on 3 Raspberry Pi 4B boards. Kubernetes Pi Cluster I chose the 4 GB model even though, according to Kubernetes Docs, 2 GB or RAM should be enough.

If You are interested in building your own Cluster, here is the list of components I used:

Step 1 - Creating the MicroSD

From https://www.raspberrypi.org/software/ download Raspberry Pi Imager

  1. Plug the MicroSD Card into you computer and launch Raspberry Pi Imager: Raspberry Pi Imager
    1. Select the OS "Raspberry Pi OS Lite" because it has no GUI
    2. Select your MicroSD Card
    3. Write the OS onto the MicroSD

Enabling ssh

Since your Raspberry Pis don't have a GUI, we'll manage them over SSH. In order to do that, we'll have to create a ssh file in the /boot partition.

  1. In Windows explorer, select the letter assiciated to your MicroSD, In my case D: Selecting the MicroSD letter
  2. Inside the folder, create a new empty txt file. Name it ssh, without extension. Creating the ssh file This will instruct the Raspberry OS to enable and start the ssh service at boot.

Step 2 - Booting the Pis and setting hostname and IP

  1. From your home router DHCP devices list, you should be able to see you Raspberry Pi IP DHCP devices list in my case, the name is raspberrypi_Ethernet and the ip 192.168.0.114.
  2. Now SSH into you Pi and set the hostname. The default username is pi and default password is raspberry: SSH to the Raspberry Pi Using an editor of your choice edit /etc/hostname and /etc/hosts with your choosen name: Editing the hostname You could also use sed to replace the original hostname with the new one:
    1$ sudo sed -i 's/raspberrypi/itdd-k8s-worker-2/' /etc/hosts /etc/hostname
    
  3. [Optional] In my case I've also decided to create a DCHP reservation, so that I'll always know what IP assigned to what device: DHCP Reservation
  4. Proceed in the same way for the remaining nodes. in my case I'll setup 3 nodes:
    • itdd-k8s-master-1, IP 192.168.0.140
    • itdd-k8s-worker-1, IP 192.168.0.151
    • itdd-k8s-worker-2, IP 192.168.0.152
  5. If everything worked fine, you should have your k8s nodes up and running: k8s nodes listed

Step 3 - Software installation

The following commands have to be exected on all the nodes

Updating the OS

  1. Upgrade the system:
    1sudo apt update && sudo apt upgrade -y
    
  2. Add the following options to /boot/cmndline.txt:
    1cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory
    

    So that the file should look like this:

    1$ cat /boot/cmdline.txt
    2console=serial0,115200 console=tty1 root=PARTUUID=f8890bb1-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory
    
  3. Uninstall and disable swap:
    1$ sudo dphys-swapfile swapoff
    2$ sudo dphys-swapfile uninstall
    3$ sudo apt purge -y dphys-swapfile
    
  4. Reboot the nodes:
    1$ sudo reboot
    

Docker installation

  1. Install Docker and add your user to the docker group
    1$ curl -sSL get.docker.com | sh
    2$ sudo usermod -aG docker pi
    

In the above, "curl -sSL get.docker.com | sh", be careful when you pipe a curl or wget command into sh, since you don't have control on what is actually been executed on your server. Only do it if you trust the source, or alternatively, download the script first and have a look at it before executing it.

  1. Logout and log back in

  2. As the root user create the file /etc/docker/daemon.json with the following content:

    1{
    2"exec-opts": ["native.cgroupdriver=systemd"],
    3"log-driver": "json-file",
    4"log-opts": { "max-size": "100m" },
    5"storage-driver": "overlay2"
    6}
    
     1$ sudo su -
     2$ cat << EOF > /etc/docker/daemon.json
     3{
     4"exec-opts": ["native.cgroupdriver=systemd"],
     5"log-driver": "json-file",
     6"log-opts": { "max-size": "100m" },
     7"storage-driver": "overlay2"
     8}
     9EOF
    10$ exit
    
  3. Create a directory for the control files:

    1$ sudo mkdir -p /etc/systemd/system/docker.service.d
    

Kubernets installation

  1. First, import the GPG key, like we did for the Docker repository: $ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
  2. Add kubernetes repositiry:
    1$ sudo vim /etc/apt/sources.list.d/kubernetes.list
    

    And add the following line to the file:

    1deb http://apt.kubernetes.io/ kubernetes-xenial main
    
  3. Install required Kubernetes packages
    1$ sudo apt update
    2$ sudo apt install -y kubeadm kubectl kubelet
    

Step 4 - Configuring k8s Cluster

Initialize the k8s Cluster

Only on the Master node:

  1. Initialize the Cluster
    1sudo kubeadm init --pod-network-cidr=10.244.0.0/16
    

    kubeadmin init

  2. run the following as a regular user:
    1mkdir -p $HOME/.kube
    2sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    3sudo chown $(id -u):$(id -g) $HOME/.kube/config
    

Deploy flannel as the CNI

In order to provide networking capabilities among the nodes, we need to deploy a Container Network Interface (CNI). There are many different options to choose from. For this instance, we are going to use one of the most common: Flannel

1kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Validate the Cluster functionality

  1. Validate all pods in the kube-system namespace are up and running: kubectl get pod --all-namespaces or kubectl get pod -n kube-system: kubectl get pod -n kube-system

Join the Worker Nodes to the Master

Recall the join command generated from the kubeadm init command and run it on each and every node as sudo: Join the worker nodes

Verify the installation from the master node

You can run kubectl get nodes to get a list of the nodes and their status: Verify the installation

You can also add -o wide to the command to get more information: -o wide view

You can also query the status of the kube-system pods as following: kubectl get pod -n kube-system -o wide

And that concludes this article on how to install and configure a 3 nodes Kubernetes Cluster on Raspberry Pi.

See Also


comments powered by Disqus