Application Migration and Disaster Recovery with Red Hat Advanced Cluster Management (ACM)

Hello! In a previous post, we covered how to use ACM for blue-green deployments, cross-cluster application migrations, and disaster recovery.

Today we will show how to use ACM to seamlessly migrate the reversewords application between our clusters and consider how to implement a basic disaster recovery scenario using PlacementRules.

Migrating Applications Using Red Hat ACM

Create new PlacementRules and Subscription

We will create two new PlacementRules, one for the clusters in the EU region and one for the US. In addition, to deploy our application in the region where it should run, we will use the new Subscription.

1. Create a new Namespace to store the required manifests.

oc --context hub create -f https://raw.githubusercontent.com/RHsyseng/acm-app-lifecycle-blog/master/acm-manifests/reversewords-region/00_namespace.yaml

2. Create the PlacementRules required to target clusters in the EU and US regions.

# PlacementRule targeting EU region clusters
oc --context hub create -f https://raw.githubusercontent.com/RHsyseng/acm-app-lifecycle-blog/master/acm-manifests/reversewords-region/01_placement_rule_EU.yaml
# PlacementRule targeting US region clusters
oc --context hub create -f https://raw.githubusercontent.com/RHsyseng/acm-app-lifecycle-blog/master/acm-manifests/reversewords-region/02_placement_rule_US.yaml

3. Create Subscription and Application.

NOTE. Here Subscription is configured to deploy the application using a PlacementRule that selects clusters in the EU region.

oc --context hub create -f https://raw.githubusercontent.com/RHsyseng/acm-app-lifecycle-blog/master/acm-manifests/reversewords-region/03_subscription-region.yaml
oc --context hub create -f https://raw.githubusercontent.com/RHsyseng/acm-app-lifecycle-blog/master/acm-manifests/reversewords-region/04_application-region.yaml

4. Now let’s make sure that our application is running on a cluster in the EU region (this is our Development cluster).

oc --context dev -n reverse-words-region get deployments,services,pods
NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.extensions/reverse-words   1/1     1            1           4m39s

NAME                    TYPE           CLUSTER-IP      EXTERNAL-IP                                                                  PORT(S)          AGE
service/reverse-words   LoadBalancer   172.30.79.111   a3115b78bce924ddc885d2b7dab766a6-1199935412.eu-central-1.elb.amazonaws.com   8080:30254/TCP   4m39s

NAME                                 READY   STATUS    RESTARTS   AGE
pod/reverse-words-68795d69ff-xmwc6   1/1     Running   0          4m39s

5. Now let’s execute the same request on a cluster in the US region (this is our Production cluster) and make sure that there are no running pods there.

oc --context pro -n reverse-words-region get deployments,services,pods
No resources found in reverse-words-region namespace.

Migrating the application

Now let’s say that for some legal reason our application can no longer run on servers in the EU region, and therefore needs to be ported to US servers. No question, this is done with just one command.

Remember we have two PlacementRules, one for EU s servers, one for US servers. So we’ll just patch our Subscription so it stops using PlacementRule for EU servers and starts using PlacementRule for US servers.

Thus, by replacing the PlacementRule that is used in our Subscription, we will automatically move the application from one region to another.

1. Patching Subscription.

The patch below rules the PlacementRule used by Subscription on clusters in the us region.

oc --context hub -n reverse-words-region patch subscription.apps.open-cluster-management.io/reversewords-region-app-subscription -p '{"spec":{"placement":{"placementRef":{"name":"us-region-clusters"}}}}' --type=merge

2. As a result, our application will be automatically migrated from EU-clusters to US-clusters, which can be tracked by the output of the commands below.

The app no ​​longer runs in the EU region (this is the development cluster).

oc --context dev -n reverse-words-region get deployments,services,pods
No resources found in reverse-words-region namespace.

It now runs in the US region (this is the production cluster).

oc --context pro -n reverse-words-region get deployments,services,pods
NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.extensions/reverse-words   1/1     1            1           92s

NAME                    TYPE           CLUSTER-IP       EXTERNAL-IP                                                              PORT(S)          AGE
service/reverse-words   LoadBalancer   172.30.177.196   a90273a7fa3ea4015989fac522b6b36e-709976322.us-west-2.elb.amazonaws.com   8080:30375/TCP   2m33s

NAME                                 READY   STATUS    RESTARTS   AGE
pod/reverse-words-68795d69ff-jlktw   1/1     Running   0          92s

As you can see, PlacementRules make it easy to move applications between clusters. We used the region as the criterion, but any other labels that are configured on the clusters can be used in the PlacementRule.

Disaster Recovery Using Red Hat ACM

Now let’s show you how to implement a basic disaster recovery scenario using PlacementRules.

Configuring the required ACM manifests

We’ll use the ACM manifests from the previous section, in other words, we’ll use a Namespace called reverse-words-region and a Subscription called reversewords-region-app-subscription.

We need to create a new PlacementRule with new properties like this:

apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
  name: us-eu-region-clusters
  namespace: reverse-words-region
spec:
  clusterConditions:
    - type: "ManagedClusterConditionAvailable"
      status: "True"
  clusterSelector:
    matchExpressions:
    - key: region
      operator: In
      values:
      - EU
      - US
    matchLabels: {}
  clusterReplicas: 1

  1. The matchExpressions property is used to select all clusters that have EU or US in their region tag.
  2. The clusterReplicas property is needed to select only one of the clusters selected in the previous paragraph.
  3. In addition, we select only clusters that are healthy.

This new PlacementRule ensures that if one of the clusters goes into the non-healthy state, then the cluster returned by the PlacementRule will change to one of the clusters in the healthy state.

Configuring Subscription to use this new PlacementRule

1. Now let’s create the PlacementRule we just talked about.

oc --context hub create -f https://raw.githubusercontent.com/RHsyseng/acm-app-lifecycle-blog/master/acm-manifests/reversewords-region/05_placement_rule_DR.yaml

If you look at the list of clusters returned by this PlacementRule, then there will be only one cluster (in our case, Production).

oc --context hub -n reverse-words-region get placementrule us-eu-region-clusters -o yaml
<OMITTED_OUTPUT>
status:
  decisions:
  - clusterName: managed-cluster1-dev
    clusterNamespace: managed-cluster1-dev

2. Go ahead and update the Subscription that we used in the previous section on migration. We’ll patch it to use the PlacementRule we just created.

oc --context hub -n reverse-words-region patch subscription.apps.open-cluster-management.io/reversewords-region-app-subscription -p '{"spec":{"placement":{"placementRef":{"name":"us-eu-region-clusters"}}}}' --type=merge

3. The application will run in the EU cluster (this is Development), which can be tracked by the output of the commands below:

oc --context dev -n reverse-words-region get deployments,services,pods
NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.extensions/reverse-words   1/1     1            1           42s

NAME                    TYPE           CLUSTER-IP      EXTERNAL-IP                                                                PORT(S)          AGE
service/reverse-words   LoadBalancer   172.30.185.94   a520ed21ff982452abeacf63b0b58cc5-31012041.eu-central-1.elb.amazonaws.com   8080:32283/TCP   42s

NAME                                 READY   STATUS    RESTARTS   AGE
pod/reverse-words-68795d69ff-crzqp   1/1     Running   0          42s

4. Now let’s kill the EU cluster to simulate an emergency and see what happens.

NOTE. We will actually kill our cluster now. If you don’t want to kill your own, then just remove the region: EU tag from it.

  1. As soon as ACM sees that the EU cluster is gone, the PlacementRule output will update to point to the US cluster.
    oc --context hub -n reverse-words-region get placementrule us-eu-region-clusters -o yaml
    

    The PlacementRule now points to the US cluster.

    <OMITTED_OUTPUT>
    status:
      decisions:
      - clusterName: managed-cluster2-prod
        clusterNamespace: managed-cluster2-prod
    

  2. The application automatically moved to the US cluster.
    oc --context pro -n reverse-words-region get deployments,services,pods
    NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
    deployment.extensions/reverse-words   1/1     1            1           76s
    
    NAME                    TYPE           CLUSTER-IP       EXTERNAL-IP                                                               PORT(S)          AGE
    service/reverse-words   LoadBalancer   172.30.187.142   a1c7d218d901c40ac98375f4a9474084-1310645059.us-west-2.elb.amazonaws.com   8080:31095/TCP   78s
    
    NAME                                 READY   STATUS    RESTARTS   AGE
    pod/reverse-words-68795d69ff-ttzz5   1/1     Running   0          77s
    

  3. The EU cluster no longer exists.

5. When the EU cluster is online again, it will be automatically added to the PlacementRule.

Conclusion

So, in part one, we covered the basic concepts of the application life cycle in Red Hat ACM and showed how to apply them by deploying an application across two clusters. Second, how to use ACM for blue-green deployments, cross-cluster application migration, and disaster recovery. And in the final part, we discussed application migration and disaster recovery using Red Hat ACM. Until next time!

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *