How we create an application based on a microservice architecture, what features we encounter and how we get around them

Hello! My name is Ivan, I am a Lead Engineer at Nexign. In this article, I would like to talk a little about the solution for supporting the operator’s business processes and revenue management and share the experience of developing one of its components using microservices. This information will be useful both for engineers who are going to use the microservice architecture in application development, and for product owners and managers who need to understand its fundamentals to assess the risks associated with the project.

What are solutions to support the operator’s business processes

First of all, let’s define what solutions are for supporting the operator’s business processes and revenue management. These are products for working with the financial objects of the customer, which include automation of billing processes and billing for services to customers based on data on payments, adjustments, aggregated information about consumption events and related charges. Revenue management solutions also include receivables automation. We architecturally singled out a separate functional block dedicated to this task, and began to build it using microservices.

Our functional block is designed to automate decision-making on the formation of control actions on external systems based on the analysis of financial data and key customer characteristics obtained from various sources. Business rules (measures) are used to achieve this goal. They are combined into work plans that implement the sequence of application of measures when the conditions specified by various parameters are met.

The functional block for automating the work with receivables includes the following functions:

  • configuration and storage of measures and work plans;

  • making decisions on the use of a particular measure with the subsequent formation of control actions for external systems;

  • providing information to third-party systems on the progress of work with debt.

As mentioned above, we decided to build our block according to the microservice architecture. The main factors in choosing this approach were the flexibility of development, management and scaling. Nevertheless, any development is associated with a number of difficulties, and I will tell you what strategies we follow to solve some of them.

Slicing microservices

The first difficulty associated with a microservice architecture is the slicing of microservices.

A microservice is a self-contained, independently deployable software component that implements certain useful functions. Its API may include commands, requests, or events. Proper decomposition into microservices and the definition of how they interact will save you from tons of burnt man-days associated with “recomposing” services in the further development of the solution.

When slicing microservices, it is worth remembering one of the main principles of the approach – the loose coupling of services. Microservices should be independent from each other. Ideally, they should not have common structures and data stores, and may also have different internal architectures, and sometimes a separate technology stack. In other words, each service should have its own API and its own database or separate schema if it has a data persistence need, and should be able to handle a limited and related set of tasks well. Slicing microservices also depends on network latencies, ensuring data consistency between services when writing, since some operations are forced to update data in several services, and obtaining a consistent view of data, because information can come from different databases.

When developing our functional block, we chose the following algorithm for slicing microservices:

  • definition of functional requirements: creation of a generalized domain model and definition of system operations;

  • distribution to services by business functions (this is not always possible due to intersections);

  • service API definition.

Based on the purpose of our functional block, we are considering the option of cutting the product into the following services:

  • services for configuring and storing profiles;

  • services for configuring and storing measures;

  • services for configuring and storing work plans;

  • service for working with external events;

  • several services that serve to execute work plans and manage NSI.

I will also illustrate the slicing of microservices using a picture:

Choosing how services interact

The second difficulty associated with microservice architecture is the correct choice of how services interact. I note right away that there is no silver bullet here: everyone chooses an approach based on individual needs and limitations.

Service interaction can be synchronous or asynchronous, and each of these methods has its pros and cons.

Synchronous communication is based primarily on RPC or REST technologies. REST is the most common, but request-response communication can limit services. In turn, RPC (if we talk about the most common implementation of gRPC) is a little more complicated, but the use of HTTP / 2 and ProtocolBuffers increases its effectiveness. One of the main disadvantages of synchronous interaction options is the need to specify the location of all services. When accessing another service, each instance of the service needs to know about the particular “location” of the neighbor, but this issue is easily resolved using serviceDiscovery. Synchronous interaction options are also associated with difficulties in the event of failures or timeouts.

In turn, asynchronous communication methods can be built on the basis of synchronous protocols using request identifiers and callbacks, but in most cases they are based on the use of message brokers. I would like to note that the correct functioning of asynchronous communication requires the definition of message types: what will you drive through the broker – entities, commands or events? Message types also influence the choice of interaction patterns, such as unidirectional notifications, publisher/subscriber, and others. The broker-based asynchronous method provides more flexible and reliable communication between services and increases system availability, but it is more difficult to implement. The message broker can also become a bottleneck, since it contains the entire system of interaction between services.

When developing our functional block, we chose a combined exchange option, trying to use an asynchronous method of interaction where it is indispensable. We are building a synchronous interaction based on RPC, and an asynchronous interaction based on Apache Kafka.

Ensuring transactionality

Another challenge associated with microservices is the provision of transactionality. In a microservice architecture, it is difficult to support ACID type transactions, as some data can be updated across multiple services, and data consistency is one of the key development points.

Let us briefly illustrate the situation. Let’s imagine that we have a “Some Action” business transaction that needs to conduct smaller transactions in other services. In this case, the entire “Some Action” action makes sense only on the condition that all child transactions are completed successfully. If at least one of them ended with an error, then you need to roll back all previous completed child transactions, since otherwise we will get an inconsistent state.

There are several ways to solve this problem, including distributed transactions and sagas (sets of local transactions), also known as narratives. We settled on the variant with sagas and an orchestrator. This method seemed to us suitable due to its simplicity and relatively inexpensive development cost.

The essence of the solution is quite simple. A business transaction is broken down into a sequence of smaller transactions in which calls are made to services responsible for local transactions. After that, a certain scenario for the execution of this transaction (saga) is compiled, which includes error handling (service failures). The task of coordinating the performance of the saga falls to the orchestrator.

Based on the requirements for different errors (failures), a specific method or sequence of processing methods is specified:

  • repetition of the appeal;

  • launching compensating transactions;

  • completion of the transaction with an error.

I would like to briefly dwell on compensatory transactions. In essence, a compensating transaction is a rollback of every local transaction that has already been made. The disadvantage of this processing method is the need to implement additional compensating methods, but they may also be absent for natural reasons.

Below is an illustration that captures the essence well on the example of booking a tour.

The final step in solving this problem is the choice of a component for the performance of the saga (orchestrator). Here you can reinvent the wheel yourself and implement something of your own, or you can consider ready-made solutions such as Camunda, Apache AirFlow and others. It is important to remember that the orchestrator/coordinator must be reliable, scalable, and able to pick up the saga from the point of failure.

When developing our functional block, we tried various ready-made tools, including Camunda, on which we write bpmn scripts as sagas, and Temporal, on which the saga is directly coded. We eventually settled on Temporal as we thought it was a production-ready solution that was fairly easy to integrate into a project. By and large, all Temporal integration comes down to marking up the code with annotations and writing a binding for its interaction with the server. I want to note that the main advantage of this solution is the ease of deployment thanks to dockerCompose for a quick start on a local machine or in a test environment and Helm charts for deploying to kubernetes. When deploying to k8s, you can also immediately get Temporal monitoring out of the box with metrics displayed in Grafana.


In this article, I have considered only the “tip of the iceberg” of application development based on microservice architecture. I would like to note that there are quite a lot of difficulties that are introduced by the use of microservices at all stages of the software life cycle, from thinking through the concept of a solution to its operation. However, they can be dealt with, and I hope this article will help you with this.

If you have questions or comments, please share them in the comments.

I would also like to emphasize again the importance of the conscious application of this approach. It is worth remembering that many tasks that are often solved using microservices are easily performed without them or are not related to them.

Even the brightest and most expensive emerald will not replace a hammer if you need to hammer a nail©.

Similar Posts

Leave a Reply