Our experience of integration with Diadoc – outgoing process architecture

Diadoc – a system of electronic document management with counterparties.

AlfaStrakhovanie interacts with counterparties, partners, agents and others through various document management systems, including Diadoc. Since electronic document management (hereinafter EDM) with counterparties in the Diadoc system corresponded well to the processes that the team was already developing, and we have experience interacting with other external systems, we decided to create an integration between the AlfaStrakhovanie electronic document management system (hereinafter EDM-AS) and the Diadoc system. This integration will be discussed in this article.

Analysis of interaction with the Diadoc system

User Interaction

The team divided interaction with the Diadoc system into two flows: incoming – receiving documents sent by the counterparty, and outgoing – sending documents to the counterparty or internal employee.

After analyzing the current interaction of AlfaStrakhovanie with the Diadoc system, we identified several major points:

1. Despite the presence various document flow formatsour employees and contractors primarily use unformalized – transfer of documents not tied to a specific format

2. In the Diadoc system, you can assign a specific employee as a document signatory

3. The employee who sends documents for signing to the Diadoc system will not necessarily be the one who signs them on behalf of AlfaStrakhovanie

4. One of the most routine parts of the work for an employee is the moment of transferring EDI files (signature files, protocols, etc.) from the Diadoc system to the SED-AS. At this point, documents are often lost

5. The party that first sends the signed document pays for EDI in the Diadoc system

6. The Diadoc system has implemented the functionality of transferring the opportunity to the counterparty to sign the document first, using sample – a pre-filled document that has no legal force and is sent without a signature

Test environment

In the Diadoc system for integrating organizations, there is no separate test environment with a host, authorization, etc. This point is played out differently. Test organizations are created in the system, each of which can be configured, simulating real ones. Within the Diadoc system, there is probably a clear division of zones into test and productive, but when interacting visually, both from the UI point of view and from the API point of view, it will seem that everything is in one contour.

Our team created several test organizations, one of which we consider AlfaStrakhovanie in the test, and the rest – as if counterparties. Various challenges were played on them API Diadoc.

EDI in the Diadoc system is a message with documents, which over time becomes overgrown with patches in the form of actions on these documents. We quickly figured out that creating EDI in Diadoc is implemented relatively simply, by calling 2 methods:

1. We put documents on the shelf in Diadoc (ShelfUpload)

2. We create a message with the posted documents (PostMessage)

If the message is not created as a Template, then we add one more action – send the message to the signatory for signature (PostMessagePatch), selected by the manager from AlfaStrakhovanie.

Implementation

Calling an integration from another team's system

As a scrum team we move in iterations.

The first type of documents that go through the integration are related to the outgoing flow, so we decided to start with implementing outgoing interaction. Another team was responsible for sending these documents.

After joint discussion, requirements were formulated in the form of user stories for integration.

• I, as the calling system, want to select a counterparty by TIN, KPP and OGRN

• I, as the calling system, want to appoint a signatory from AlfaStrakhovanie by his email

• I, as a calling system, want to know about the completion of the life cycle of documents in the Diadoc system

• I, as the calling system, want documents to be loaded into the SED-AS upon completion of their life cycle in the Diadoc system

I will say right away that in the following architectural drawings I will omit previously drawn details and display only the changed elements to simplify the perception of the pictures.

At the beginning of the development, the question arose: “Is it necessary to duplicate information about organizations in Diadoc?” As you can see from the figure, we decided that it was necessary. Our integration periodically queries the Diadoc API. It takes information about our counterparties and their divisions from there. It also takes information about users registered as AlfaStrakhovanie employees in the Diadoc system. This solution allows us not to request this information every time the user needs it. As a bonus, it gives the opportunity to use the advantages of our directories built on OData. The Diadoc API allows us to quickly receive information about new relationships with counterparties, thanks to which we set up synchronization as follows:

• Employee data once per hour. We implement it using the method in the API GetEmployees

• Data on our new counterparties and departments in them every 3 minutes. We implement it using the method in the API GetCounteragents using afterIndexKey

• Data on all our counterparties and departments in them once a day. We implement using methods in the API GetCounteragents And GetCounteragent

This synchronization allowed us to minimize the time lag in updating the directory so much that users of the ECM-AS do not notice it.

In the case of an outgoing process, the start occurs from the AlfaStrakhovanie circuit, which significantly simplifies the interaction architecture.

As you can see from the figure, the process of sending documents for signing is not complicated. An external team sends a request to our service. The service searches for the required counterparty based on the data sent, creates a message with documents and sends it to Diadoc for signing. After that, it puts information about the created message into the database (hereinafter DB) for further tracking.

The diagram also shows the main disadvantage of such interaction. After creating a message to track the completion of EDI documents, you need to periodically poll the proxy service. It is much better if the integration itself reports the completion upon the occurrence of this event.

I can highlight one advantage: gateways (intermediate services) were created to interact with the Diadoc system. This allowed us to set up communication with the Diadoc API on the side of the main services in the form we are accustomed to. This also turned out to be useful in situations where information security officers want to monitor traffic in a special way, and monitoring methods require special measures for the service.

Since the implementation period is short, we did not bother with tracking the statuses of the EDI either. As can be seen from the figure, essentially we created one background service that once in a certain period of time reads messages from the DB, queries Diadoc on them and, if the documents have completed statuses, loads the EDI files into the SED-AS and archives the message record in the DB.

The main disadvantage of this implementation is the formation of a queue of monitored messages in the DB table. Firstly, this approach does not scale – you have to keep one instance of the application working with the table. Secondly, the Diadoc API has a method GetDocflowEvents. This method allows you to query Diadoc only for changed messages, and not for everything in the table. We plan to switch to it, as it provides a higher speed of synchronization of document statuses at lower costs.

Calling integration from SED-AS

Now it was necessary to enrich the existing solution so that it could be used directly from the SED-AS. For this story, the set of requirements was not much different:

• As a manager, I want to send documents for signing to Diadoc without leaving SED-AS

• As a manager, I want to see in SED-AS the statuses of the EDI sent to Diadoc

• As a manager, I want to appoint a signatory from AlfaStrakhovanie

• As a manager, I want to send documents to a specific department of a counterparty

• As a manager, I want the EDI files (signature files, protocols, etc.) to be loaded into the SED-AS by themselves.

If we move away from the requirements, we wanted to build approximately the following picture. The manager opens the ECM-AS, selects a counterparty and a signatory from AlfaStrakhovanie. Then sends the documents to the Diadoc system for signing. While the manager is busy with other matters, the integration pulls up the EDI status in the ECM-AS. And as soon as the life cycle of the documents in the Diadoc system is completed (they are signed or rejected), the integration will load all the necessary information on EDI into the ECM-AS. From this moment, the manager will be able to see using the usual tools in the ECM-AS that work on his documents in Diadoc has been completed, which will speed up decision-making on his further actions.

To send documents from SED-AS to the Diadoc system, we created a special form where the manager can select the items he needs. Earlier, we decided to duplicate information about counterparties and signatories in our directories, which simplified the implementation of the output of suggestions for auto-filling fields when typing characters.

You can also notice that we do not use words like Template on the form. It seemed that this would confuse the user, and we simply wrote which party would be the first to sign documents in EDI.

There have also been changes in the architecture of interaction with integration. It so happened that between the previous and this implementation, interaction through corporate Kafka was actively developing in the company. And our team learned how to use it. Therefore, we decided to add its use to the integration. Now, when sending a document for signing, the topic is specified Kafka. When the life cycle is completed, information will be sent to it via EDI.

Starting a process from the EDM-AS has its own specific features, so after completing the EDM, we added the service “Synchronization Service for Process-Specific Data in the EDM-AS”, which takes them into account.

Result

Integration is gradually gaining momentum in the company, but there is still room for growth. To date, more than 200,000 documents have been sent through integration into the Diadoc system.

Since the data was collected in mid-August 2024, it is not complete for the 3rd quarter of 2024. But as can be seen from the graph, the main trend is towards an increase in the number of EDIs created through integration. This is a natural picture, because over time, more and more different teams in our company are connecting to sending documents through our integration.

Conclusion

Based on our experience in creating various integrations to external systems, we can say that Diadoc has a fairly rich and good API, although not all methods are obvious and clear at first glance. It has sufficient functionality to create an integration that automates the processes of the Operational Block in terms of interaction with Diadoc.

I would like to add that the documentation of the Diadoc system is at a good level, and its API contains interesting solutions that I can adopt for future projects. Well, and it is simply interesting to observe how much the interactions of different external systems with which our ECM-AS needs to be friends can differ.