Application Migration and Disaster Recovery with Red Hat Advanced Cluster Management (ACM)
Today we will show how to use ACM to seamlessly migrate the reversewords application between our clusters and consider how to implement a basic disaster recovery scenario using PlacementRules.
Migrating Applications Using Red Hat ACM
Create new PlacementRules and Subscription
We will create two new PlacementRules, one for the clusters in the EU region and one for the US. In addition, to deploy our application in the region where it should run, we will use the new Subscription.
1. Create a new Namespace to store the required manifests.
oc --context hub create -f https://raw.githubusercontent.com/RHsyseng/acm-app-lifecycle-blog/master/acm-manifests/reversewords-region/00_namespace.yaml
2. Create the PlacementRules required to target clusters in the EU and US regions.
# PlacementRule targeting EU region clusters
oc --context hub create -f https://raw.githubusercontent.com/RHsyseng/acm-app-lifecycle-blog/master/acm-manifests/reversewords-region/01_placement_rule_EU.yaml
# PlacementRule targeting US region clusters
oc --context hub create -f https://raw.githubusercontent.com/RHsyseng/acm-app-lifecycle-blog/master/acm-manifests/reversewords-region/02_placement_rule_US.yaml
3. Create Subscription and Application.
NOTE. Here Subscription is configured to deploy the application using a PlacementRule that selects clusters in the EU region.
oc --context hub create -f https://raw.githubusercontent.com/RHsyseng/acm-app-lifecycle-blog/master/acm-manifests/reversewords-region/03_subscription-region.yaml
oc --context hub create -f https://raw.githubusercontent.com/RHsyseng/acm-app-lifecycle-blog/master/acm-manifests/reversewords-region/04_application-region.yaml
4. Now let’s make sure that our application is running on a cluster in the EU region (this is our Development cluster).
oc --context dev -n reverse-words-region get deployments,services,pods
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.extensions/reverse-words 1/1 1 1 4m39s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/reverse-words LoadBalancer 172.30.79.111 a3115b78bce924ddc885d2b7dab766a6-1199935412.eu-central-1.elb.amazonaws.com 8080:30254/TCP 4m39s
NAME READY STATUS RESTARTS AGE
pod/reverse-words-68795d69ff-xmwc6 1/1 Running 0 4m39s
5. Now let’s execute the same request on a cluster in the US region (this is our Production cluster) and make sure that there are no running pods there.
oc --context pro -n reverse-words-region get deployments,services,pods
No resources found in reverse-words-region namespace.
Migrating the application
Now let’s say that for some legal reason our application can no longer run on servers in the EU region, and therefore needs to be ported to US servers. No question, this is done with just one command.
Remember we have two PlacementRules, one for EU s servers, one for US servers. So we’ll just patch our Subscription so it stops using PlacementRule for EU servers and starts using PlacementRule for US servers.
Thus, by replacing the PlacementRule that is used in our Subscription, we will automatically move the application from one region to another.
1. Patching Subscription.
The patch below rules the PlacementRule used by Subscription on clusters in the us region.
oc --context hub -n reverse-words-region patch subscription.apps.open-cluster-management.io/reversewords-region-app-subscription -p '{"spec":{"placement":{"placementRef":{"name":"us-region-clusters"}}}}' --type=merge
2. As a result, our application will be automatically migrated from EU-clusters to US-clusters, which can be tracked by the output of the commands below.
The app no longer runs in the EU region (this is the development cluster).
oc --context dev -n reverse-words-region get deployments,services,pods
No resources found in reverse-words-region namespace.
It now runs in the US region (this is the production cluster).
oc --context pro -n reverse-words-region get deployments,services,pods
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.extensions/reverse-words 1/1 1 1 92s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/reverse-words LoadBalancer 172.30.177.196 a90273a7fa3ea4015989fac522b6b36e-709976322.us-west-2.elb.amazonaws.com 8080:30375/TCP 2m33s
NAME READY STATUS RESTARTS AGE
pod/reverse-words-68795d69ff-jlktw 1/1 Running 0 92s
As you can see, PlacementRules make it easy to move applications between clusters. We used the region as the criterion, but any other labels that are configured on the clusters can be used in the PlacementRule.
Disaster Recovery Using Red Hat ACM
Now let’s show you how to implement a basic disaster recovery scenario using PlacementRules.
Configuring the required ACM manifests
We’ll use the ACM manifests from the previous section, in other words, we’ll use a Namespace called reverse-words-region and a Subscription called reversewords-region-app-subscription.
We need to create a new PlacementRule with new properties like this:
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
name: us-eu-region-clusters
namespace: reverse-words-region
spec:
clusterConditions:
- type: "ManagedClusterConditionAvailable"
status: "True"
clusterSelector:
matchExpressions:
- key: region
operator: In
values:
- EU
- US
matchLabels: {}
clusterReplicas: 1
- The matchExpressions property is used to select all clusters that have EU or US in their region tag.
- The clusterReplicas property is needed to select only one of the clusters selected in the previous paragraph.
- In addition, we select only clusters that are healthy.
This new PlacementRule ensures that if one of the clusters goes into the non-healthy state, then the cluster returned by the PlacementRule will change to one of the clusters in the healthy state.
Configuring Subscription to use this new PlacementRule
1. Now let’s create the PlacementRule we just talked about.
oc --context hub create -f https://raw.githubusercontent.com/RHsyseng/acm-app-lifecycle-blog/master/acm-manifests/reversewords-region/05_placement_rule_DR.yaml
If you look at the list of clusters returned by this PlacementRule, then there will be only one cluster (in our case, Production).
oc --context hub -n reverse-words-region get placementrule us-eu-region-clusters -o yaml
<OMITTED_OUTPUT>
status:
decisions:
- clusterName: managed-cluster1-dev
clusterNamespace: managed-cluster1-dev
2. Go ahead and update the Subscription that we used in the previous section on migration. We’ll patch it to use the PlacementRule we just created.
oc --context hub -n reverse-words-region patch subscription.apps.open-cluster-management.io/reversewords-region-app-subscription -p '{"spec":{"placement":{"placementRef":{"name":"us-eu-region-clusters"}}}}' --type=merge
3. The application will run in the EU cluster (this is Development), which can be tracked by the output of the commands below:
oc --context dev -n reverse-words-region get deployments,services,pods
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.extensions/reverse-words 1/1 1 1 42s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/reverse-words LoadBalancer 172.30.185.94 a520ed21ff982452abeacf63b0b58cc5-31012041.eu-central-1.elb.amazonaws.com 8080:32283/TCP 42s
NAME READY STATUS RESTARTS AGE
pod/reverse-words-68795d69ff-crzqp 1/1 Running 0 42s
4. Now let’s kill the EU cluster to simulate an emergency and see what happens.
NOTE. We will actually kill our cluster now. If you don’t want to kill your own, then just remove the region: EU tag from it.
- As soon as ACM sees that the EU cluster is gone, the PlacementRule output will update to point to the US cluster.
oc --context hub -n reverse-words-region get placementrule us-eu-region-clusters -o yamlThe PlacementRule now points to the US cluster.
<OMITTED_OUTPUT> status: decisions: - clusterName: managed-cluster2-prod clusterNamespace: managed-cluster2-prod - The application automatically moved to the US cluster.
oc --context pro -n reverse-words-region get deployments,services,pods NAME READY UP-TO-DATE AVAILABLE AGE deployment.extensions/reverse-words 1/1 1 1 76s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/reverse-words LoadBalancer 172.30.187.142 a1c7d218d901c40ac98375f4a9474084-1310645059.us-west-2.elb.amazonaws.com 8080:31095/TCP 78s NAME READY STATUS RESTARTS AGE pod/reverse-words-68795d69ff-ttzz5 1/1 Running 0 77s - The EU cluster no longer exists.
5. When the EU cluster is online again, it will be automatically added to the PlacementRule.
Conclusion
So, in part one, we covered the basic concepts of the application life cycle in Red Hat ACM and showed how to apply them by deploying an application across two clusters. Second, how to use ACM for blue-green deployments, cross-cluster application migration, and disaster recovery. And in the final part, we discussed application migration and disaster recovery using Red Hat ACM. Until next time!
