Istio Ambient Mesh Performance in One Vs. several Kubernetes namespaces
There are many articles about Service Mesh and Istio, including Ambient Mesh, the newest candidate for Service Mesh in Kubernetes. This post is about my experience evaluating the performance of Istio's Ambient Mesh, simulating an enterprise-grade deployment across multiple Kubernetes namespaces.
Istio and Ambient Mesh
For those unfamiliar with Service Mesh, it is a special infrastructure layer in Kubernetes that allows communication between services to be more secure, faster, and more reliable. Typically, Service Meshes are built with a two-tier architecture containing a Control Plane to configure the network and a Data Plane to provide network functionality. The Data Plane consists of a sidecar container that is attached to the microservice pod and consumes additional CPU and memory resources.
Although this sidecar architecture is already well known, the Service Mesh community of developers and users has shown great interest in improving its resource consumption. For this purpose the company solo.io, with experience in Service Mesh, has created a new Data Plane architecture called Ambient Mesh. Google later joined the project and an open source version was made available to us as part of Istio's Service Mesh. If you're interested in reading about the new architecture, the Istio blog has some very good starting points.
Setting up test infrastructure
GCP remains my top cloud provider choice for this evaluation with demo microservices application Bank of Anthos (BOA) from Google. Setting up the test environment involves initializing the Kubernetes cluster and Grafana visualization stack using Terraform scripts, and then installing Istio. Microservices were then deployed using Helm charts, including binding the frontend microservice to the Istio ingress controller. This makes it possible to access the application from outside the cluster, which allows you to perform load testing from a local workstation.
Two BOA deployment scenarios are used to perform memory and CPU efficiency tests in two Istio modes.
First, we will look at a typical example of a small organization, where 8 microservices of a BOA application are deployed in one Kubernetes namespace with two replicas.
This will be followed by an enterprise deployment model, in which microservices are typically deployed in different namespaces with team-level access limited by Kubernetes RBAC policies. In our case, RBAC is not used, since it does not affect the performance of the Service Mesh, but BOA microservices are deployed in 8 different namespaces with 6 replicas.
The purpose of this test is to test both Istio modes in a real deployment environment.
An interesting factor is the impact of the performance of the Waypoint proxy server in Ambient mode, since the cluster has 8 Waypoint proxy servers connected to each namespace. In both cases, the number of Ztunnel proxies varied depending on the number of nodes in the cluster.
The following figure shows a visualization of the GCP infrastructure for BOA deployments across multiple namespaces. The diagram shows the deployment of up to 3 namespaces to reduce the complexity of the figure, however, in the real test, 8 namespaces are used, each of which represents one microservice. When deployed across multiple “Istio-System” and “Monitoring” namespaces, the namespaces remain unchanged, but the Data Plane Istio components change slightly. Ztunnel is deployed one at a time on each node, while a Waypoint proxy instance is used multiple times to cover multiple namespaces.
Efficiency mark
Benchmark metrics are sourced from Grafana and display the minimum, maximum, and average CPU and memory consumption over a 10-minute time period for each test. This evaluation includes 12 different test situations, covering single and multi-namespace environments.
The performance difference was assessed by comparing average CPU and memory usage between Istio sidecar mode (default configuration) and Ambient mode (with L4 and L4+L7 proxy enabled). Two different comparisons are made between sidecar mode and Ambient mode in two different configurations to track the resource consumption of the Ztunnel and Waypoint proxies.
As a result, Istio's CPU utilization improved by 7.4% in the single-namespace test scenario and by 39% in the multi-namespace test scenario.
Memory usage by the Istio system also showed a significant improvement in Ambient mode: 79.45% in one namespace and 74.98% in different namespaces.
Conclusion
The sidecarless Ambient Mesh model demonstrates noticeable performance advantages over the sidecar model and saves on cloud resources. Looking at the test results, Ambient Mesh can be considered as the future of Service Mesh, enabling non-invasive design. If you are interested in setting up Ambient Mesh in your Kubernetes cluster, visit istio.io there is a guide. Also, if you'd like to take a look at the scripts used for this evaluation, feel free to take a look at my Github repository.
