5G Core Network Deployment with Free5GC, Kubernetes and Helm

Hello everyone, dear Khabrovites. I continue translating articles about 5G and the practical implementation of a demo to explore the capabilities and architecture of 5G. In the previous article, we took a look at 5G architecture and building a 5G core network with Open5GS and UERANSIM.

Foreword

In my previous post, I talked about deploying a 5G core network using Open5GS and setting up a 5G UE and 5G RAN simulator using UERANSIM. In this post, I will talk about deploying a Free5GC based 5G core network using Kubernetes and Helm. Service monitoring with Prometheus/Grafana. All deployments related to this post are available at gitlab. Please clone the repository and continue the post.

Free5GC

Free5GC is an open source project for 5G mobile core networks. It is the world’s first open source 5G core network that complies with international standards 3GPP Release 15 (R15). The original goal of Free5GC was to provide a platform for scientists to test and prototype 5G systems. However, due to its completeness and open source, Free5GC also has commercial value, especially for private 5G networks.

A basic 5G network based on Free5GC can be deployed with Kubernetes using Helm charts. Orange Open Source provides open source Helm charts for deploying Free5GC with Kubernetes. Following are the basic steps you need to follow to deploy a Free5GC 5G network on Kubernetes.

Kubernetes Configuration

I am running a single node Minikube Kubernetes cluster on an AWS Ubuntu 20.04 server. I had to use 4 CPUs, 8 GB of memory for the Kubernetes cluster to run the Free5GC services (otherwise the products might start reporting insufficient CPU or memory resources).

To manage networking in a Kubernetes cluster, I used the Calico Container Network Interface (CNI) plugin. CNI supports plugin-based features to simplify networking in Kubernetes. By default, Kubernetes uses the Kubenet plugin for networking (for example, processing incoming/outgoing requests). Kubenet is a very simple plugin that doesn’t have many features. If we need additional features such as cross-domain isolation, IP filtering, traffic mirroring, or changing load balancing algorithms, then other network plugins should be used. Various CNI plugins are available: Flannel, Calico, WeaveNet, Cilium, Canal. In this scenario, I used the Calico CNI plugin. Using Calico, I assigned static IPs to the pods, enabled SCTP traffic on the cluster, etc.

# install required packages
sudo apt update -y
sudo apt upgrade -y
sudo apt install -y curl wget apt-transport-https

# install minikube
wget https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo cp minikube-linux-amd64 /usr/local/bin/minikube
sudo chmod +x /usr/local/bin/minikube

# start minikube
# use calico as cni plugin
minikube start --driver=docker --cpus=4 --memory=8g --disk-size=20g --cni=calico

# verify minikube installation
❯❯ minikube status
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured


----


# install kubectl
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
chmod +x kubectl
sudo mv kubectl /usr/local/bin/

# verify kubectl installation
❯❯ kubectl version -o yaml
clientVersion:
  buildDate: "2021-12-16T11:41:01Z"
  compiler: gc
  gitCommit: 86ec240af8cbd1b60bcc4c03c20da9b98005b92e
  gitTreeState: clean
  gitVersion: v1.23.1
  goVersion: go1.17.5
  major: "1"
  minor: "23"
  platform: linux/amd64
serverVersion:
  buildDate: "2021-10-27T18:35:25Z"
  compiler: gc
  gitCommit: c92036820499fedefec0f847e2054d824aea6cd1
  gitTreeState: clean
  gitVersion: v1.22.3
  goVersion: go1.16.9
  major: "1"
  minor: "22"
  platform: linux/amd64
  
    
---


# install helm
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

# verify helm installation
❯❯ helm list -A
NAME	NAMESPACE	REVISION	UPDATED	STATUS	CHART	APP VERSION

Enabling the Multus-CNI Plugin

Multis-CNI is a CNI plug-in for Kubernetes that allows you to connect multiple network interfaces to pods. Typically, in Kubernetes, each pod has only one network interface (besides the loopback). With Multus you can create a multi-homed module with multiple interfaces. This is achieved by having Multus act as a meta plugin, a CNI plugin that can call several other CNI plugins.

To run Multus-CNI, you first need to install the Kubernetes CNI plugin to serve networking between pods, I used the Calico CNI plugin. The Calico CNI plugin creates a default network interface that each module will be created with. Every network connection created by Multus will be added to this network interface by default.

# install multus-cni
cat ./deployments/multus-daemonset-thick-plugin.yml | kubectl apply -f -

# verify installation
❯❯ kubectl get pods --all-namespaces
kube-system   calico-kube-controllers-58497c65d5-vz6pf   1/1     Running   0             17h
kube-system   calico-node-4b44m                          1/1     Running   0             17h
kube-system   kube-multus-ds-94kkw                       1/1     Running   0             17h

Helm Chart Deployment

After setting up the K8s cluster and CNI, I can deploy Free5GC 5G core network services with Helm Chart. I used the Free5GC Helm Chart provided by Orange Open Source.

# create namespace 
kubectl create namespace free5gc

# add helm repository
helm repo add towards5gs 'https://raw.githubusercontent.com/Orange-OpenSource/towards5gs-helm/main/repo/'
helm repo update

# view repository list
❯❯ helm repo list
NAME      	URL
towards5gs	https://raw.githubusercontent.com/Orange-OpenSource/towards5gs-helm/main/repo/

# view available charts
❯❯ helm search repo
NAME                          	CHART VERSION	APP VERSION	DESCRIPTION
towards5gs/free5gc            	1.0.0        	v3.0.5     	A Helm chart to deploy Free5gc
towards5gs/free5gc-amf        	0.1.1        	v3.0.6     	A Helm chart to deploy the Free5GC AMF
towards5gs/free5gc-ausf       	0.1.1        	v3.0.6     	A Helm chart to deploy the Free5GC AUSF
towards5gs/free5gc-n3iwf      	0.1.1        	v3.0.6     	A Helm chart to deploy the Free5GC N3IWF
towards5gs/free5gc-nrf        	0.1.1        	v3.0.6     	A Helm chart to deploy the Free5GC NRF
towards5gs/free5gc-nssf       	0.1.1        	v3.0.6     	A Helm chart to deploy the Free5GC NSSF
towards5gs/free5gc-pcf        	0.1.1        	v3.0.6     	A Helm chart to deploy the Free5GC PCF
towards5gs/free5gc-smf        	0.1.1        	v3.0.6     	A Helm chart to deploy the Free5GC SMF
towards5gs/free5gc-udm        	0.1.1        	v3.0.6     	A Helm chart to deploy the Free5GC UDM
towards5gs/free5gc-udr        	0.1.1        	v3.0.6     	A Helm chart to deploy the Free5GC UDR
towards5gs/free5gc-upf        	0.1.1        	v3.0.6     	A Helm chart to deploy the Free5GC User Plane
towards5gs/free5gc-webui      	0.1.1        	v3.0.6     	A Helm chart to deploy the Free5GC WEBUI
towards5gs/free5gcControlPlane	0.1.2        	v3.0.5     	DEPRECATED - A Helm chart to deploy the control...
towards5gs/free5gcN3iwf       	0.1.2        	v3.0.5     	DEPRECATED - A Helm chart to deploy the n3iwf f...
towards5gs/free5gcUserPlane   	0.1.2        	v3.0.5     	DEPRECATED - Helm chart to deploy the user plan...
towards5gs/networks5g         	0.1.2        	0.1.2      	DEPRECATED - A Helm chart to deploy the user pl...
towards5gs/ueransim           	2.0.12       	v3.2.4     	A Helm chart to deploy UERANSIM


---


# deploy helm
# -n free5gc(namespace)
# free5gc-v1(name of the helm deployment)
# towards5gs/free5gc(chart name)
helm -n free5gc install free5gc-v1 towards5gs/free5gc

# verify helm deployment
❯❯ kubectl get pods -n free5gc
NAME                                              READY   STATUS              RESTARTS   AGE
free5gc-v1-free5gc-amf-amf-6f896894fd-9kvd8       0/1     Pending             0          36s
free5gc-v1-free5gc-ausf-ausf-64d6cc5bbb-2n4qb     0/1     Init:0/1            0          36s
free5gc-v1-free5gc-nrf-nrf-7d89776696-vpq9t       0/1     Init:0/1            0          36s
free5gc-v1-free5gc-nssf-nssf-79cd7cb489-dtpnx     0/1     Running             0          36s
free5gc-v1-free5gc-pcf-pcf-796f7f6ccd-s8d2g       0/1     Running             0          36s
free5gc-v1-free5gc-smf-smf-6994dff84d-qs7rx       0/1     Running             0          36s
free5gc-v1-free5gc-udm-udm-7997c46d89-jw8jh       0/1     Running             0          36s
free5gc-v1-free5gc-udr-udr-655664c8d8-fcs4d       0/1     Running             0          36s
free5gc-v1-free5gc-upf-upf-7b9c645bfb-bhjdg       0/1     Running             0          36s
free5gc-v1-free5gc-webui-webui-76b8fb879f-n47zm   0/1     Running             0          36s
mongodb-0                                         1/1     Running             0          36s

Free5GC Web UI

Free5GC provides a web interface for setting up UE devices and other configurations on the 5G core network. The web interface is represented by a Kubernetes service with nodePort=30500. I can access it using this URL {replace-by-the-IP-of-one-of-your-cluster-nodes}:30500 or via Kubernetes port forwarding. In the web interface, I can register UE devices. I have deployed basic 5G services on AWS. In order to access the web interface service from my local machine, I performed a port forward for SSH.

# web ui service run port 5000 as NodePort service(on aws install)
❯❯ kubectl get svc -n free5gc
NAME            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
webui-service   NodePort    10.98.157.168   <none>        5000:30500/TCP   4m43s

# kubernets port forward to access the NodePort service
kubectl port-forward --namespace free5gc svc/webui-service 5000:5000

# do ssh port forwarding to access the web-ui service on aws from local machine
# now web ui can access in your local machine localhost:5000
ssh -L localhost:5000:localhost:5000 ubuntu@3.137.120.206

I can then register the subscriber (UE device) via the web interface. Read more about UE device configuration in the web interface in my previous post.

Service monitoring

I used Prometheus to monitor core 5G services in Kubernetes. Visualization made with Grafana. prometheus-community provides a Helm Chart for installing Prometheus/Grafana services. Following are some of the services available in the prometheus community.