Saga pattern as a way to ensure data consistency

Hello. Already in OTUS opens a set for a new course group “Highload Architect”… In this regard, I continue a series of my publications written specifically for this course, and also invite you to my free demo lesson on the topic: “Indexes in MySQL: best practices and pitfalls.” You can sign up for the webinar here…

Introduction

As you know, the transition from a monolith to a microservice architecture causes a number of difficulties associated with both the technical part of the project and the human factor. One of the most difficult technical challenges is the provision of consistency in a distributed system.

Last time we discussed the causes of consistency problems in a microservice architecture, an optimistic approach to ensuring consistency, and ensuring consistency using a two-phase commit.

Saga pattern

Saga is a mechanism for ensuring data consistency in a microservice architecture without using distributed transactions.

For each system command that needs to update data in several services, a saga is created. The saga is a kind of “checklist” consisting of sequential local ACID transactions, each of which updates data in one service. A compensating transaction is applied to handle failures. Such transactions are executed in case of failure on all services on which local transactions have completed successfully.

There are several types of transactions in the saga, as many as four:

Compensating – Cancels a change made by a local transaction.
A refundable is a transaction that needs to be refunded (canceled) in the event that subsequent transactions fail.
Turning – a transaction that determines the success of the entire saga. If it succeeds, then the saga is guaranteed to reach the end.
Repeatable – goes after the pivot and is guaranteed to succeed.

You can organize a saga with choreography or orchestration.

In the case of the choreographic saga, there is no dedicated orchestrator. Using the order service and users as an example, it might look like this: the order service receives a request and creates an order in the PENDING state, and then publishes the “Order created” event. An event handler in the user service processes this event, tries to reserve an item, and publishes the result as an event. The order service processes this event, confirming or canceling the order depending on the read result.

The orchestrated saga looks a little more interesting. Using the above services as an example, it might look like this: the order service receives a request, creates a saga that creates an order in the PENDING state, and then sends a command to reserve goods for the user service. The user service tries to reserve the product and sends a response message indicating the result. The saga approves or cancels the order.

The saga pattern allows an application to maintain data consistency across multiple services without using distributed transactions (two-phase commits) and avoiding the problems discussed in the previous article. But on the other hand, the programming model is greatly complicated: for example, the developer for each transaction must write a compensating transaction that reverses the changes made earlier within the saga.

The saga allows us to achieve an ACD model (Atomicity + Consistency + Durability in ACID terms), but we have lost one letter. The lack of the letter I leads to the well-known problems of lack of isolation. These include: lost updates – one saga overwrites the changes made by another without reading them, dirty reads – a transaction or saga reads unfinished updates from another saga, fuzzy / nonrepeatable reads) – two different stages of the saga read the same data, but get different results because another saga made changes. There are a number of patterns that allow you to fix certain anomalies: semantic locking, commutative updates, pessimistic representation, re-reading of a value, file of changes and by value. The issue of ensuring isolation remains open.

Another interesting issue is the impossibility of atomic updating the database and posting a message to the message broker to trigger further steps in the saga.

Conclusion

We talked about ways to organize a saga using choreography and orchestration, as well as about the problems that this pattern entails. Next, we will talk about ways to fix some anomalies and transactionally send messages to the message broker.