As promised in first part case, we talk about the stack used by the IT team of Sberbank when working on a project with big data. For those who have not read it: thanks to this project, the developers have learned how to effectively interact with the “business”, going through crisis points. But in this part, we will not talk about optimizing the work of the IT department, but about the technical underside of the project.
Let us remind you that in this blog we tell our potential colleagues about what projects, technologies and teams are now in Sberbank and what awaits you if you join us. Therefore, today at the end of the technical case, we will tell you a little about the IT team itself, in case you are thinking about working with us.
Sber’s IT team has developed an internal online monitoring system that allows you to see the entire history of requests from a legal entity client and promptly respond to his questions. This system unites in text format all the client’s inbox, scattered across the communication channels: mail, instant messengers, telephone conversations.
The project uses only one PostgreSQL database instance. Namely: Postgres Sber Edition (developed by Sber), which the team decided to use as part of import substitution. At the same time, it was necessary to see how this database would work with highly loaded data and whether it could replace Spark, Hadoop, Kafka Streams and other tools that companies usually use to work with big data. Plus, this solution allowed the team to quickly roll out the MVP of the system, which, we recall, was supposed to be the first on the market – which means it needs to work quickly.
What the online monitoring system consists of and what is the main principle of its operation
The system consists of two parts:
Workplace of a monitoring service employee
Workplace of a monitoring service employee
It displays a tag cloud for a certain period of time (for example, a day). In tags, the most frequently used meaningful words increase in size, so you can understand which task requires urgent attention (for example, the words “credit” or “penalty”).
In order for the operator to be able to understand whether the situation develops into an incident, he has a traffic light indicator: green – everything is fine; yellow – a significant word has begun to be used more often, possibly the beginning of an incident; red – too frequent use, incident.
To “catch” incidents, a screen was created where you can see all the dialogues between clients and operators at a given time. Incident dialogues are highlighted. An employee of the monitoring service can connect to the dialogue and watch it in text format literally by the participants’ remarks.
In the tag cloud, you can click on all large words, and so go to the list of all dialogues with clients, where this word appears more often than others – this way you can quickly answer questions about a problem that has become typical at the moment (for example, some some function of the application), as well as find patterns in the shortcomings of customer service by the bank.
The core of the application is built on a microservice architecture. It integrates with various bank systems:
Avaya telephony for tracking telephone conversations between customers and agents.
System SmartSpeech (transcriber, development of the company SberDevices), which processes audio traffic and transcribes it into text using AI natural language processing (NLP) and the Kafka distributed message broker. The transcribed text is saved to the database.
Integration platform Synapse (event segment, development SberTech) to get data through Kafka.
An application instance connects to the database, searches for the necessary data and sends it to Python modules with AI models. Models process the data, then put it back into the database.
How is the transcription of telephone conversations
The AES (Application Enablement Services) integration module connects to Avaya telephony via a special GET ip interface and collects all information about a telephone call at the very beginning. At the same moment, the module sends the call metadata to the transcriber SmartSpeechwhich is also connected to Avaya. This is where the process of transcribing text starts: the transcriber receives RTP traffic from Avaya in the form of sound and converts it into printed text using a special analysis tool. After that SmartSpeech sends the text to the corporate service bus of Sberbank, from where it goes to the compilation module of our system, which saves the data to the database. All this happens in real time, which imposes certain difficulties.
SmartSpeech begins to transcribe phrases at the moment when long pauses appear between them (just like a person, he “understands” by a pause that the phrase is complete). Kafka discards transcribed messages, and a service specially developed by the IT team reads them in multithreaded mode in milliseconds. This allows you to watch the dialogue online in real time. With the help of such a system, you can learn a lot about clients (we will tell you about what exactly you can learn and how to use this knowledge in the next article about the project).
What is interesting and at the same time difficult to work on this project
The team has developed both a custom application (frontend) and a core (backend), which is integrated with many bank services and is the nodal link of a highly loaded system. Through this system, there is a constant stream of big data, which is not just transcribed and displayed. Dialogues from this data are processed by artificial intelligence models, that is, the system allows you to analyze and store the analytical data of the dialogues, so that they can then be effectively used for marketing purposes.
You know, perhaps it’s time to stop being modest and say that the project uses the most advanced technologies and programs from today’s IT world: microservices, containers, AI, Big Data, speech technologies, OpenShift, Kafka, Java Spring and others. Effective global development methodologies (DevOps, Agile, Git-flow) are fully used, and the team is trying to use the approach of combining different paths with one goal.
The good news is that as a result of optimizing the work of the IT department, which we talked about in the first part, developers have the opportunity to master these technologies, try to squeeze the maximum out of them, test their versions. Sberbank’s developers argue from their previous experience that few Russian companies devote so much time to a thorough study of modern high technologies. At the same time, there are no delays in development – release cycles coincide with two-week sprints. And one of the developers who came to work for us from Europe says that it was only in Sberbank that he found such an approach to interaction in the IT team, which he was taught at a European university and which is adopted in advanced world companies.
In conclusion, I would like to give a little explanation about the specifics of working in the Sberbank IT team. Some time ago, the Sberbank Corporate Architecture Department underwent fundamental changes that support the new goals of the entire ecosystem: the introduction of advanced IT products and services, large-scale innovations, and competition with global technology companies. Thanks to this vector, developers at Sberbank have the opportunity to use almost any technology, try any methods. Well, this makes working on such complex projects in Sberbank interesting and truly creative!