explain with an example

Namespaces are a critical resource for supporting a multi-tenant Kubernetes cluster architecture. But they are difficult to manage when working with large-scale multi-tenant clusters. Fortunately, the process can be simplified by adding the hierarchical namespaces feature to Kubernetes. Team VK Cloud translated an article on how to do it.

This article is for those who have a general understanding of Kubernetes, clusters, containers, and pods. You will also need basic knowledge of namespaces in Kubernetes.

Kubernetes Namespaces 101

Namespaces are a Kubernetes resource that allows you to isolate other types of resources from each other. As with most Kubernetes resources, you can create a space in two ways: define it in a manifest file using a declarative approach, or imperatively, on the command line. Below is an example of the manifest file code for creating the coolapp space. This is a declarative approach:

apiVersion: v1
kind: Namespace
  name: coolapp

The following set of commands imperatively creates a namespace using the kubectl CLI tool:

kubectl create namespace coolapp

Once a namespace is created, engineers assign other Kubernetes resources to it. The example below shows how to imperatively run under Kubernetes with an nginx container and assign it the coolapp space displayed as a result of the command.

# kubectl run ngnix --image nginx -n coolapp pod/ngnix created

To enumerate resources in this namespace, it must be declared. Here is the code for getting pods into the coolapps namespace and the results of its execution:

# kubectl get pods -n coolapp
ngnix   1/1     Running   0          3m5s

If a resource is assigned to a namespace, then it can only be accessed through that namespace. This code shows the result of getting pods in the default namespace, which in this case has no pods:

# kubectl get pods
No resources found in default namespace.

Why couldn’t the pods be found? Because the one created earlier under nginx is assigned to the coolapp namespace and not the default namespace. Under nginx, it is only visible in the coolapp space.

Security as a Namespace Benefit

If a resource is only available in a specific namespace, this increases its security level. In Kubernetes namespaces with

role-based access control

(RBAC) you can assign access rights to users, groups, and other resources. For example, an engineer can create a coolapp-admins group and give it full access to any resources in the coolapp namespace. Full access means that users in the coolapp-admins group can create, list, update, fix, and delete resources.

On the other hand, an engineer can create a coolapp-devs group that only has permissions to create, update, and delete pods in the coolapp namespace. The rights of this group concern only pods and certain actions that can be performed with them.
Very fine granularity is maintained when assigning permissions to resources and roles in namespaces. This level of security comes in very handy when supporting a multi-tenant architecture.

Fundamentals of Kubernetes multi-tenant architecture

A multi-tenant Kubernetes cluster is a cluster where multiple organizations coexist (see Figure 1 below).

Kubernetes can run multiple tenants in the same cluster

A multi-tenant architecture is more cost-effective and easier to manage than having separate clusters for each department or organization.

In a dedicated architecture, each organization has its own Control Plane and virtual machines with worker nodes. This is a rather expensive solution. In addition, dedicated technical staff will be required for each cluster. All this can lead to high excess costs. And if the resources are not used in full, it’s just a waste of money.

On the other hand, since all operations are concentrated in a single cluster in a multi-tenant architecture, the efficiency of the use of physical resources and labor costs of IT specialists is increased.

Simplifying multi-tenant namespace management

In a multi-tenant Kubernetes cluster, organization isolation is implemented using namespaces. As we wrote above, in a single Kubernetes cluster, using space, you can separate the access rights of users, groups, and resources. Thus, if engineers need to allocate resources to a tenant in a Kubernetes cluster, they assign a tenant to one or more namespaces in that cluster. Resource permissions are granted in a specific space (see Figure 2 below).

In a multi-tenant cluster, tenants are separated from each other using Kubernetes namespaces

As an example, imagine a multi-tenant cluster with three tenants: companies A, B, and C. Each of these companies has tenants that are allowed to use the notional Acme-App shared enterprise application, which is hosted on the same cluster.

Now let’s imagine that in each company there are several departments that have the right to use different features of the application. In our example, these would be HR, Finance, and Planning.

So, if three companies need to be created and isolated from each other in a Kubernetes cluster, engineers create a set of namespaces:


To implement permissions, it was necessary to create nine namespaces for tenants. This is a perfectly suitable structure, which, however, can turn into a headache for technical support. For example, if engineers need to change a permission that is valid for all divisions of the same company, they will need to make changes to at least three namespaces. And if you want to change the permission for all companies using Acme-App, then you have to assign permissions for all nine spaces. Details can be found

from a video dedicated to this topic


Obviously, a more rational approach is needed. And these are just hierarchical namespaces. If they are implemented in a Kubernetes cluster, you can create a structure like this:

├── company-a
│   ├── company-a-hr
│   ├── company-a-finance
│   └── company-a-scheduling
├── company-b
│   ├── company-b-hr
│   ├── company-b-finance
│   └── company-b-scheduling
├── company-c
│   ├── company-c-hr
│   ├── company-c-finance
│   └── company-c-scheduling

As you can see, hierarchical namespaces implement a simplified organizational structure. In addition to organizational simplicity, such spaces allow you to distribute permissions to subordinate namespaces.

Let’s return to the acme-app tree from the illustration above. If a permission is changed in the acme-app root namespace, the change will propagate to all sub-namespaces in the tree structure. And if engineers need to make a change to all of company A’s sub-namespaces, all they need to do is change the permissions on company A’s namespace. The sub-namespaces company-a-hr, company-a-finance, and company-a-scheduling will adjust automatically.

Hierarchical spaces not only make it easier to manage tenants in a multi-tenant Kubernetes cluster, they also increase operational efficiency in terms of overall operations and security. Unfortunately, at the time of this writing, hierarchical namespaces are not shipped out of the box with Kubernetes, they must be installed.

Adding Hierarchical Namespaces to a Kubernetes Cluster

Installing hierarchical namespaces in a Kubernetes cluster is a simple matter. Below is an example code that does this under Linux. In addition, the script installs the hns plugin for the kubectl command line tool. The plugin and kubectl are designed to create and maintain hierarchical namespaces. Example taken from

official instructions

VK Cloud will also work.

kubectl label ns kube-system hnc.x-k8s.io/excluded-namespace=true --overwrite
kubectl label ns kube-public hnc.x-k8s.io/excluded-namespace=true --overwrite
kubectl label ns kube-node-lease hnc.x-k8s.io/excluded-namespace=true --overwrite

This is necessary in order to eliminate problems with service spaces.

kubectl apply -f https://github.com/kubernetes-sigs/hierarchical-namespaces/releases/download/${HNC_VERSION}/default.yaml 
HNC_PLATFORM=linux_amd64 # также поддерживаются: darwin_amd64, darwin_arm64, windows_amd64
curl -L https://github.com/kubernetes-sigs/hierarchical-namespaces/releases/download/${HNC_VERSION}/kubectl-hns_${HNC_PLATFORM} -o ./kubectl-hns
chmod +x ./kubectl-hns
kubectl hns

Bash script to set hierarchical namespaces in Kubernetes cluster

After the script is executed, the Kubernetes cluster will start supporting hierarchical namespaces. We can now run some more code to create the hierarchical namespaces for the acme-app application that we’ve already featured in our article.

# Root namespace

kubectl create namespace acme-app
sleep 2  # Take a rest so the namespace can provision

# Company A namespaces
kubectl hns create company-a -n acme-app
sleep 2  # Take a rest so the namespace can provision
kubectl hns create company-a-hr -n company-a
kubectl hns create company-a-finance -n company-a
kubectl hns create company-a-scheduling -n company-a

# Company B namespaces
kubectl hns create company-b -n acme-app
sleep 2 # Take a rest so the namespace can provision

kubectl hns create company-b-hr -n company-b
kubectl hns create company-b-finance -n company-b
kubectl hns create company-b-scheduling -n company-b

# Company C namespaces
kubectl hns create company-c -n acme-app
sleep 2 # Take a rest so the namespace can provision

kubectl hns create company-c-hr -n company-c
kubectl hns create company-c-finance -n company-c
kubectl hns create company-c-scheduling -n company-c

# Display the hierarchical namespaces tree
kubectl hns tree acme-app

Script to create a set of hierarchical namespaces for the acme-app multi-tenant application

This is how the tree created as a result of executing the above code looks like:

├── [s] company-a
│   ├── [s] company-a-finance
│   ├── [s] company-a-hr
│   └── [s] company-a-scheduling
 ── [s] company-b
│   ├── [s] company-b-finance
│   ├── [s] company-b-hr
│   └── [s] company-b-scheduling
└── [s] company-c
    ├── [s] company-c-finance
    ├── [s] company-c-hr
    └── [s] company-c-scheduling

[s] indicates subnamespaces

The namespace hierarchy created by the script reflects the hierarchy we introduced earlier in this article.

Summing up

Hierarchical namespaces offer clear benefits when running multiple tenants on the same Kubernetes cluster. Thanks to them, you can manage permissions in cascade mode. Any change to the permissions that are relevant to the parent namespace will apply to the child namespaces as well. The efficiency is obvious.

As we already mentioned, hierarchical namespaces need to be set for each cluster. They are not implemented in the standard version of Kubernetes. You will need to build the process of preparing them into your company’s deployment workflow. This will take some effort, but it’s worth it. If your company supports multi-tenant Kubernetes clusters, then it makes sense to make hierarchical spaces a standard component of the Kubernetes architecture: in the long run, this will save you time and money.

Try Kubernetes as a Service on the VK Cloud platform. We give new users 3000 bonus rubles for testing the cluster or any other services.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *