Kubernetes best practices. Kubernetes Zero Downtime Cluster Upgrade

Kubernetes best practices. Create small containers
Kubernetes best practices. Kubernetes organization with namespace
Kubernetes best practices. Kubernetes viability test with Readiness and Liveness tests
Kubernetes best practices. Setting Queries and Resource Limits
Kubernetes best practices. Correct Terminate Disable
Kubernetes best practices. Mapping external services

Everyone knows how good it is to keep your apps up to date. Kubernetes and Docker can make the upgrade process a lot easier, so you can create a new container with updated dependencies and easily deploy it. In addition to updating application dependencies, Kubernetes is constantly updating features and security policies.
Thus, the base nodes and infrastructure of Kubernetes must be up to date. In this series, we will learn how the Google Kubernetes Engine can easily upgrade the Kubernetes cluster.

So, when updating a cluster, you need to update the wizards and nodes, and the wizards must be updated first. Let’s see how to update both elements using the Google Kubernetes Engine. This system automatically updates the wizard as it releases point releases. However, as a rule, it will not be updated to the new version, for example, 1.7-1.8, automatically. When you are ready to upgrade to the new version, you can simply click the Upgrade available link in the GKE console, after which a dialog box will appear on the screen. It contains a warning that changing the master version may take several minutes, during which you cannot edit this cluster, while during the upgrade of the wizard your deployments and services will continue to work normally. However, the entire infrastructure that needs the Kubernetes API will not work.

This means that QPTL and all applications that use the Kubernetes API to obtain cluster information will be disabled and you will not be able to make any changes to the cluster.

Let’s see how you can solve this problem by updating the wizard with zero downtime.

While the standard GKE zone cluster supports only one wizard, you can create regional clusters that provide multi-zone high availability wizards. Therefore, when creating your cluster, be sure to choose a regional option.

Your nodes and wizards will be automatically created in three zones, and the wizards will have load-balanced IP addresses. This will keep Kubernetes API running smoothly during the upgrade.

When updating nodes, there are several different strategies that you can use. I want to focus your attention on two things – rolling rolling updates and migration using node pools.

The easiest way to update Kubernetes nodes is to use rolling update, the default update mechanism that GKE uses to update your nodes. It works as follows.

The nodes of the old version are decommissioned one after another so that all modules cease to work in them. Then these nodes are deleted, and instead of them, new nodes of the updated Kubernetes version appear one after another. After one node starts working, the other proceeds to the update process, and this continues until all nodes are updated. You can let GKE manage this process for you by turning on automatic updating of nodes in the node pool by selecting Enabled.

If you do not, the GKE dashboard will warn you when a new update is available. In this case, to perform the update, you need to click the Automatic node updates link and follow the instructions.

At the same time, it is very important to make sure that your pods are controlled using the replica set, a stateful set, or something similar. Then the autonomous hearths will not be restructured.

Although rolling updates are pretty easy to do with GKE, it still has a few drawbacks. One of them is that when you upgrade, the capacity of your cluster decreases by one node. This disadvantage is easily eliminated by scaling the node pool by adding additional capacity and then reducing it after the update.

In addition, the fully automated nature of rolling updates makes updating easier, but leaves you with less control over the process. If something goes wrong and you have to roll back to the old version, it will take time to stop rolling updates and discard any changes that have already been made. Let’s see how you can use pools from multiple nodes to upgrade your cluster.

So, instead of updating the active pool of nodes using rolling updates, you create a completely new pool of nodes, wait for all nodes to start, and then transfer workloads one node at a time. As you execute these commands yourself, you gain more control over the migration process, while GKE continues to manage your nodes.
Suppose a Kubernetes cluster consists of 3 virtual machines. You can view the nodes using the get nodes command.

To create a new pool of nodes named pool-two, you need to use the appropriate command, setting it up in exactly the same way as the command for the old pool.

Optionally, you can also use the GUI to create a new node pool. For more information on this, use the Node pool creation link located below this video.

If you check the number of nodes again, you will find three new nodes with the pool-two pool name, however, the pods will still be on the old nodes.

Let’s move them to a new node pool, moving one node at a time in rolling mode. To do this, use the cordon command for each old node to protect them and prevent the formation of new hearths in them.

As soon as all old nodes are fenced, the creation of hearths will be planned only in new nodes. This means that you can start removing pods from old nodes, and Kubernetes will automatically plan to create them in new nodes. Then you need to “drain” each node, which will lead to the removal of all hearths in the node.

After you do this for one node, make sure that the new pods are ready and already working, and then move on to the next node. If you had any problems during the migration, run uncordon for the old pool, and then run cordon and drain for the new pool. In this case, the pods will be automatically transferred back to the old pool. Once all the pods have been safely transferred, you can delete the old pool. To do this, replace the default pool with the pool you want to delete.

The Google Kubernetes Engine lets you keep your Kubernetes cluster up to date with just a few clicks. I highly recommend using GKE regional clusters for high availability wizards and automatic node updates to ensure correct and trouble-free updates.

If you need additional control over the node update process, you can provide it with the help of pools, without giving up the advantages of the management platform that GKE provides.

If you are not using GKE, then use the rolling update rolling method or node pool nodes to update the nodes of your cluster. But remember that in this case you need to manually add new nodes to the cluster and perform critical updates yourself, which may not be quite simple.

To be continued very soon …

A bit of advertising 🙂

Thank you for staying with us. Do you like our articles? Want to see more interesting materials? Support us by placing an order or recommending to your friends, cloud VPS for developers from $ 4.99, A unique analogue of entry-level servers that was invented by us for you: The whole truth about VPS (KVM) E5-2697 v3 (6 Cores) 10GB DDR4 480GB SSD 1Gbps from $ 19 or how to divide the server? (options are available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).

Dell R730xd 2 times cheaper at the Equinix Tier IV data center in Amsterdam? Only here 2 x Intel TetraDeca-Core Xeon 2x E5-2697v3 2.6GHz 14C 64GB DDR4 4x960GB SSD 1Gbps 100 TV from $ 199 in the Netherlands! Dell R420 – 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB – from $ 99! Read about How to Build Infrastructure Bldg. class c using Dell R730xd E5-2650 v4 servers costing 9,000 euros for a penny?

Similar Posts

Leave a Reply Cancel reply