Increased usage of multiple containers with microservices led to the need for a proper way to manage these containers. This is where Kubernetes comes in. Kubernetes is an open-source orchestration tool that is used to run, deploy and manage containerized workloads. It is highly available, highly scalable, and has disaster recovery methods of backup and restoring data.

k3s is a certified distribution of Kubernetes that contains all processes that run on Kubernetes packages in a single binary node. It is widely used in local development environments, testing environments, experimenting and Learning when getting started on Kubernetes.

Features

  • Lightweight and uses fewer system resources.
  • Compatibility and highly available.
  • Uses most common databases like SQLite as the default, MySQL, and PostgreSQL.
  • Eay to install and quick to launch.
  • All dependencies are bundled up in one single binary.

k3s can be used in edge computing and embedded systems, Internet of Things (IoT), Continuous Intergarteion environments, and Single-App clusters.

This guide will show you how to install Kubernetes Cluster on CentOS 9|AlmaLinux 9|RHEL 9 using k3s.

Install k3s on CentOS 9|AlmaLinux 9|RHEL 9

Update your system packages.

sudo dnf update -y

Update the crypto policies and set them to LEGACY to ensure compatibility while installing k3s on CentOS 9|AlmaLinux 9|RHEL 9.

sudo update-crypto-policies --set LEGACY

Install Master Node

You can use the following command to install the latest version.

curl -sfL https://get.k3s.io/ | sh -s - --write-kubeconfig-mode 644

This command installs additional utilities including kubectl and uninstallation script for k3s.

You can check the version to confirm successful installation

$ k3s --version
k3s version v1.23.8+k3s1 (53f2d4e7)
go version go1.17.5

Check the status of the service.

$ sudo systemctl status k3s
● k3s.service - Lightweight Kubernetes
     Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: d>
     Active: active (running) since Wed 2022-06-29 22:18:33 EAT; 1min 9s ago
       Docs: https://k3s.io
    Process: 46130 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --q>
    Process: 46132 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, statu>
    Process: 46133 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/S>
   Main PID: 46139 (k3s-server)
      Tasks: 109
     Memory: 1.3G
        CPU: 26.763s
     CGroup: /system.slice/k3s.service

It is recommended to turn off firewalld

sudo systemctl disable firewalld --now

Check to see if your master node is running.

$ kubectl get nodes
NAME                          STATUS   ROLES                             AGE   VERSION
cent9.technixleo.com   Ready    control-plane,master   31m   v1.23.8+k3s1

Use this command to get the token that we will use to install k3s on the worker node.

$ sudo cat /var/lib/rancher/k3s/server/node-token
K1007bf13277668f468616325b2311c53bf709d1f554f8012fb3f3ad3bcb6b64db2::server:1590974bb49259c934751d2defdd94bc

The K3S_TOKEN in this case is the last part of the token 1590974bb49259c934751d2defdd94bc.

Install Worker Node

Use the following command syntax to install. K3S_URL is used to make the K3s run in the worker node. The KRS_TOKEN is stored at /var/lib/rancher/k3s/server/node-token on your server node.

curl -sfL https://get.k3s.io | K3S_URL=https://[master-host]:6443 K3S_TOKEN=mynodetoken sh -

So for my installation script, I will define them as shown below.

curl -sfL https://get.k3s.io | K3S_URL=https://192.168.200.41:6443 K3S_TOKEN="1590974bb49259c934751d2defdd94bc" sh -

You can also define the fields as shown below and use a shorter installation script.

k3s_url="https://192.168.200.41:6443"
k3s_token="1590974bb49259c934751d2defdd94bc"
curl -sfL https://get.k3s.io | K3S_URL=${k3s_url} K3S_TOKEN=${k3s_token} sh -

Sample installation output

Complete!
[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Creating /usr/local/bin/ctr symlink to k3s
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-agent-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s-agent.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s-agent.service
[INFO]  systemd: Enabling k3s-agent unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s-agent.service → /etc/systemd/system/k3s-agent.service.

Start and enable the service on boot.

sudo systemctl start k3s-agent

Verify installation

$ sudo systemctl status k3s-agent
● k3s-agent.service - Lightweight Kubernetes
     Loaded: loaded (/etc/systemd/system/k3s-agent.service; enabled; vendor pre>
     Active: active (running) since Thu 2022-06-30 00:07:55 EAT; 5min ago
       Docs: https://k3s.io
   Main PID: 3338 (k3s-agent)
      Tasks: 8
     Memory: 234.0M
        CPU: 2.417s
     CGroup: /system.slice/k3s-agent.service
             └─3338 "/usr/local/bin/k3s agent"

Verify Cluster Status

Check the cluster status using the master node.

$ kubectl config get-cluster
NAME
default

$ kubectl get nodes
NAME                      STATUS   ROLES                       AGE        VERSION
almalinux.technixleo.com   Ready    <none>                      9m45s    v1.23.8+k3s1
cent9.technixleo.com         Ready    control-plane,master   4h54m     v1.23.8+k3s1
rhel9.technixleo.com          Ready    <none>                23s       v1.23.8+k3s1

$ kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:6443
CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy

$ kubectl get namespaces
NAME                 STATUS   AGE
default                 Active   115m
kube-system        Active   115m
kube-public          Active   115m
kube-node-lease   Active   115m

$ kubectl get endpoints -n kube-system
NAME             ENDPOINTS                                                   AGE
kube-dns         10.42.0.2:53,10.42.0.2:53,10.42.0.2:9153     116m
metrics-server                                                                     116m
traefik          10.42.0.8:8000,10.42.0.8:8443                         115m

$ kubectl get pods -n kube-system
NAME                                                    READY   STATUS      RESTARTS   AGE
local-path-provisioner-6c79684f77-j75ll    1/1       Running        0              117m
helm-install-traefik-crd-p2hdr                   0/1      Completed     0              117m
helm-install-traefik-2cxdn                         0/1      Completed     2              117m
svclb-traefik-599d01f7-czgzc                    2/2       Running        0              117m
coredns-d76bd69b-8sphx                         1/1       Running        0              117m
metrics-server-7cd5fcb6b7-6tlv5               0/1       Running        0              117m
traefik-df4ff85d6-fm5tg                            1/1       Running        0              117m

Enable Addons

You can enable addons on k3s using Helm. Helm is a package management tool that provides templating syntax for Kubernetes YAML manifest documents.

To install Helm using automation script run the commands below:

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

You can also download the latest version from Github, the unpack it using the following command

$ tar -zxvf helm-*-linux-amd64.tar.gz
linux-amd64/
linux-amd64/helm
linux-amd64/LICENSE
linux-amd64/README.md

Move the helm binary files to a directory on PATH.

sudo mv linux-amd64/helm /usr/local/bin/helm

Add the Helm Chart repository to install applications using Helm

helm repo add stable https://charts.helm.sh/stable
helm repo update

With Helm installed, you can search, find and install Kubernetes packages. You can browse all packages at the Artifact Hub or you can also use the following command to list packages.

helm search hub <package name>

Example of PostgreSQL package. This lists all repositories available to install PostgreSQL.

$ helm search hub postgresql
URL                                               	                CHART VERSION	APP VERSION                                      DESCRIPTION                                       
https://artifacthub.io/packages/helm/wiremind/p...	8.7.0         	11.7.0                                            	DEPRECATED Chart for PostgreSQL, an object-rela...
https://artifacthub.io/packages/helm/authorizat...	10.1.1       	11.10.0                                           	Chart for PostgreSQL, an object-relational data...
https://artifacthub.io/packages/helm/riftbit/po...	       10.11.0      	11.13.0                                           	Chart for PostgreSQL, an object-relational data...
https://artifacthub.io/packages/helm/taalhuizen...	10.1.1       	11.10.0                                           	Chart for PostgreSQL, an object-relational data...
https://artifacthub.io/packages/helm/bitnami/po...	11.6.11      	14.4.0                                            	PostgreSQL (Postgres) is an open source object-...
https://artifacthub.io/packages/helm/bitnami-ak...	11.6.10      	14.4.0                                            	PostgreSQL (Postgres) is an open source object-...
https://artifacthub.io/packages/helm/wallet-com...	10.1.1       	11.10.0                                           	Chart for PostgreSQL, an object-
...
...

You can add the repository of the maintainer you wish to use.

$ helm repo add bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories

You can list the charts available to install.

$ helm search repo bitnami
NAME                                        	CHART VERSION APP VERSION  	DESCRIPTION                                       
bitnami/airflow                             	12.5.10      	2.3.2         	Apache Airflow is a tool to express and execute...
bitnami/apache                              	9.1.11       	2.4.54       	Apache HTTP Server is an open-source HTTP serve...
bitnami/argo-cd                             	3.4.4         	2.4.3         	Argo CD is a continuous delivery tool for Kuber...
...
...
bitnami/postgresql                          	11.6.11      	14.4.0       	PostgreSQL (Postgres) is an open source object-...
bitnami/postgresql-ha                       	9.1.9        	        14.4.0       	This PostgreSQL cluster solution includes the P...
bitnami/prestashop                          	15.2.9       	1.7.8-6      	PrestaShop is a powerful open source eCommerce ...
bitnami/pytorch                             	2.4.12       	1.12.0       	PyTorch is a deep learning platform that accele...
...

To install it, use the following commands. These commands for installation can also be found at the Artifact Hub. The below command installs the package named postgresql.

$ helm install postgresql bitnami/postgresql
NAME: postgresql
LAST DEPLOYED: Thu Jun 30 12:06:54 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: postgresql
CHART VERSION: 11.6.11
APP VERSION: 14.4.0

** Please be patient while the chart is being deployed **

PostgreSQL can be accessed via port 5432 on the following DNS names from within your cluster:

    postgresql.default.svc.cluster.local - Read/Write connection

To get the password for "postgres" run:

    export POSTGRES_PASSWORD=$(kubectl get secret --namespace default postgresql -o jsonpath="{.data.postgres-password}" | base64 -d)

To connect to your database run the following command:

    kubectl run postgresql-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/postgresql:14.4.0-debian-11-r3 --env="PGPASSWORD=$POSTGRES_PASSWORD" \
      --command -- psql --host postgresql -U postgres -d postgres -p 5432

    > NOTE: If you access the container using bash, make sure that you execute "/opt/bitnami/scripts/postgresql/entrypoint.sh /bin/bash" in order to avoid the error "psql: local user with ID 1001} does not exist"

To connect to your database from outside the cluster execute the following commands:

    kubectl port-forward --namespace default svc/postgresql 5432:5432 &
    PGPASSWORD="$POSTGRES_PASSWORD" psql --host 127.0.0.1 -U postgres -d postgres -p 5432

List all releases using the following command.

$ helm list
NAME      	NAMESPACE  REVISION	             UPDATED                                	STATUS  	CHART             	APP VERSION
postgresql	default  	     1       	 2022-06-30 12:06:54.528887056 +0300 EAT	deployed	postgresql-11.6.11	14.4.0     

To uninstall a chart, use the following command syntax.

helm delete postgresql

To delete PersistenctVolumeClaim (PVC) of the package, use the following command. This will however delete al the PostgreSQL data.

kubectl delete pvc -l release=postgresql

Deploy an Application on k3s

There are a number of ways to deploy services outside a cluster.

  • NodePort
  • Cluster IP
  • Load Balancer
  • Ingress Controller

We are going to use the Nginx Ingress controller. We can install Nginx web-proxy using Helm

$ helm install nginx-ingress stable/nginx-ingress --namespace kube-system --set defaultBackend.enabled=false
NAME: nginx-ingress
LAST DEPLOYED: Thu Jun 30 12:10:51 2022
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
...
...

Check the status of Ingress controller installed.

$ kubectl --namespace kube-system get services -o wide -w nginx-ingress-controller
NAME                        TYPE              CLUSTER-IP      EXTERNAL-IP   PORT(S)                                  AGE   SELECTOR
nginx-ingress-controller   LoadBalancer   10.43.224.213   <pending>     80:31162/TCP,443:32472/TCP   19m   app.kubernetes.io/component=controller,app=nginx-ingress,release=nginx-ingress

Check to verify the application has been installed successfully.

$ kubectl get pods -n kube-system -l app=nginx-ingress -o wide
NAME                                        READY    STATUS    RESTARTS      AGE     IP               NODE                   NOMINATED NODE   
nginx-ingress-controller-5f7ff98f5-9dtd7   0/1     Running   3 (10s ago)   2m43s   10.42.0.11   cent9.technixleo.com   <none>           

Check if the services have been deployed.

$ kubectl get services

NAME            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
kubernetes      ClusterIP   10.43.0.1       <none>        443/TCP    117m
postgresql-hl   ClusterIP   None            <none>           5432/TCP   11m
postgresql      ClusterIP   10.43.242.136   <none>        5432/TCP   11m

$ kubectl get pods -o wide
NAME           READY   STATUS    RESTARTS   AGE           IP           NODE                   NOMINATED NODE   READINESS GATES
postgresql-0   1/1     Running           0          12m   10.42.0.10   cent9.technixleo.com   <none>           <none>

$ kubectl get svc -o wide
NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE    SELECTOR
kubernetes      ClusterIP   10.43.0.1         <none>         443/TCP    119m   <none>
postgresql-hl   ClusterIP   None                <none>          5432/TCP   13m    app.kubernetes.io/component=primary,app.kubernetes.io/instance=postgresql,app.kubernetes.io/name=postgresql
postgresql      ClusterIP   10.43.242.136   <none>           5432/TCP   13m    app.kubernetes.io/component=primary,app.kubernetes.io/instance=postgresql,app.kubernetes.io/name=postgresql

Uninstall k3s

To uninstall k3s on the master node, use the uninstall script as shown below.

$ /usr/local/bin/k3s-uninstall.sh
Removed:
  k3s-selinux-1.1-1.el8.noarch                                                    

Complete!
+ rm -f /etc/yum.repos.d/rancher-k3s-common.repo
+ remove_uninstall
+ rm -f /usr/local/bin/k3s-uninstall.sh

To uninstall k3s on the worker node, use the following command.

$ /usr/local/bin/k3s-agent-uninstall.sh
Complete!
+ rm -f /etc/yum.repos.d/rancher-k3s-common.repo
+ remove_uninstall
+ rm -f /usr/local/bin/k3s-agent-uninstall.sh

Remove configuration files manually by using the following command on both the master and worker node.

sudo rm -rf /var/lib/rancher/

Conclusion

k3s is a production-ready certified distribution of Kubernetes that is lightweight and packaged as a single binary. From this guide, we have Installed Kubernetes Cluster using k3s on CentOS 9|AlmaLinux 9|RHEL 9. k3s is optimized for both ARM64 and ARMv7 with supported binaries and multi-arch images for both.

Other guides to check out:

3 COMMENTS

LEAVE A REPLY

Please enter your comment!
Please enter your name here