Hardening Kubernetes with Kyverno, RuntimeClass, and Kata Containers

As more organizations adopt Kubernetes, security becomes one of their top concerns. Kubernetes has many built-in security features, but ensuring that security policies are properly enforced in a large and complex cluster can be challenging. At the same time, Kubernetes must provide the flexibility developers need to work effectively. In this article, we will look at three technologies that will help you achieve this balance: Kyverno Kubernetes Policy Engine, Kubernetes RuntimeClass and Kata Containers.

Kyverno Policy Engine

Policy engines are a powerful tool that Kubernetes uses to manage and enforce policies across the entire cluster. Policy engines allow cluster administrators to configure and enforce them for a wide range of activities, including network access, resource usage, and pod placement. Examples of policy engines are Open Policy Agent (OPA) and Kyverno.

Kyverno is a Kubernetes policy engine that allows you to apply policies to your Kubernetes clusters. With Kyverno, you can create policies that restrict access to resources, enforce naming conventions, restrict network access, and more. You can also define policies that are triggered on certain events, such as when a new deployment is created. When a policy violation occurs, Kyverno can take automatic action to fix the problem, such as blocking the deployment or removing the pod.

One of the benefits of Kyverno is that it integrates seamlessly with Kubernetes, so you can define policies using familiar Kubernetes objects like ConfigMaps and CRD. [Custom Resource Definition — специальный ресурс в Kubernetes, который позволяет вносить любые данные]. Kyverno also supports policy validation and generation, so you can easily make sure your policies work as expected and generate them based on existing Kubernetes resources.

RuntimeClasses in Kubernetes

RuntimeClasses

RuntimeClasses

RuntimeClass is a Kubernetes feature that allows you to specify a container runtime for your workloads. With RuntimeClass you can choose between different container runtimes or different configurations of the same runtime. This is useful for security because different runtimes have different security properties. For example, you can use a lightweight runtime for some workloads that require fast startup times and a more secure runtime (but with additional overhead) for workloads that require a higher level of isolation.

Another benefit of RuntimeClass is that it allows you to switch runtimes without changing workloads. For example, if you need to move to a more secure runtime, you can simply update the RuntimeClass in the workload deployment manifest and Kubernetes will automatically use the new runtime associated with that RuntimeClass.

Overall, the RuntimeClass provides a powerful tool for managing the runtime environment of Kubernetes workloads, providing greater control over security and flexibility.

Containers

Kata container runtime

Kata container runtime

Kata Containers is an open source project that provides a lightweight virtualization layer for isolation pods Kubernetes from the host system. Standard container runtimes such as “runc” provide cgroup isolation and share the same core with all pods. On the other hand, the Kata runtime (depicted as containerd-shim-kata-v2 in the above diagram) provides an additional layer of isolation by creating a dedicated virtual machine for each Kubernetes Pod. In the diagram above, it is depicted as a Pod VM. Each pod uses a separate kernel.

Benefits of using Policy Engines, RuntimeClasses and Kata together

While Policy Engines, RuntimeClasses, and Kata Containers are powerful on their own, they become even more powerful when used together. Here are some benefits of using Policy Engines, RuntimeClasses and Kata Containers together:

Administrators can use Policy Engines, RuntimeClasses, and Kata Containers to automatically ensure that workloads from specific users, or workloads that request privileged capabilities, run using the Kata container runtime. This is possible because the Kyverno policy engine automatically adds the correct RuntimeClass to the Pod definition before deploying it.

RuntimeClasses and Kata containers allow administrators to select different container runtimes, providing more flexibility in managing pods. This flexibility allows administrators to choose the runtime and container technology that best suits their specific needs, including security requirements, without sacrificing developer productivity.

The developer or user does not need to worry about RuntimeClasses or add special entries to the Pod definition. The user continues to deploy pods as usual. The magic happens behind the scenes with the policy engine.

In the next section, we’ll walk you through configuring Kyverno and look at some examples of policies using RuntimeClass and Kata.

Setup and examples

Let’s say you have a live Kubernetes cluster with helm configured. Install the Kyverno policy engine using the instructions below.

helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update
helm install kyverno kyverno/kyverno -n kyverno - create-namespace - set replicaCount=1

Note that you can use any policy engine that provides a mutation allow controller.

Below is an example of a policy that allows Pods to run in a specific namespace using the Kata container runtime. The RuntimeClass that was configured to use the Kata container runtime is named “kata” (surprise ;)):

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: run-pod-using-kata
spec:
  background: false
  rules:
  - name: "Run pods in specific namespace using Kata containers runtime"
    match:
      any:
      - resources:
          kinds:
          - Pod
          namespaces:
          - test
     mutate:
      patchStrategicMerge:
        spec:
          +(runtimeClassName): kata

Deploy the policy:

kubectl apply -f pod-policy.yaml

The following is an example policy that is guaranteed to run privileged pods or pods that require a “root” user in all namespaces except “kube-system” using the Kata container runtime:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: run-priv-pod-using-kata
spec:
  background: false
  rules:
  - name: "Run privilege pods using Kata containers runtime"
    match:
      all:
      - resources:
          kinds:
          - Pod    
    exclude:
        any:
        - resources:
            namespaces:
            - kube-system
    preconditions:
     any:
      - key: "{{request.object.spec.containers[?securityContext.privileged] | length(@)}}"
        operator: Equals
        value: 1
      - key: "{{request.object.spec.containers[?securityContext.runAsRoot] | length(@)}}"
        operator: Equals
        value: 1
    mutate:
      patchStrategicMerge:
        spec:
          +(runtimeClassName): kata

See the official documentation for more examples. Kyverno.

Conclusion

Kubernetes has revolutionized the way we deploy and manage software applications. However, securing Kubernetes clusters can be challenging. Policy Engines and RuntimeClasses are two powerful features that Kubernetes provides to improve cluster security.

By combining them with additional container runtimes such as Kata, you can create a highly secure and flexible Kubernetes environment. Kyverno enforces security policies, RuntimeClass determines the runtime for your workloads, and Kata Containers provides an extra layer of security by isolating pods from each other and from the host system. This approach allows you to maintain a high level of security without sacrificing the flexibility that developers need to work effectively.


In conclusion, we invite you to open class, dedicated to the architecture of solutions based on K8s. At the webinar, we will look at kubernetes from an architectural point of view, various types of implementation (Selfhosted, SaaS and PaaS), strengths and weaknesses of solutions, implementation examples.

You can sign up for an open class on the page of the online course “Infrastructure platform based on Kubernetes”.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *