Good work not only wolves die, but recognition engines appear

Today we, Smart Engines, we will tell the story of how we worked during the pandemic and what we got out of it. We will tell you when and for whom the size of the application matters, what is the liveliness of documents and where this wonderful world of recognition is heading.

The story of how we actually restarted Smart Engines in the grocery plan. And this story will not at all be about how we attracted billions of dollars in investments through a “penny” rebranding or successfully grabbed government subsidies on a distance basis to support the business. No, we are by nature not one of those who, as they say, “eat a fish and don’t go into a pond”. The reason for many months of hard work was a complete rethinking of the concept, architecture and purpose of our recognition engines. But first things first.

Five years ago we entered the market with a simple product – a passport recognition engine on mobile devices. The functionality was extremely simple: you show the engine the passport in the video stream, and it returns the document attributes in text form (and it did this without any connection with external servers, all recognition was carried out directly on the device). No reconciliation with the databases of valid passports, no automatic requests to the Ministry of Internal Affairs for fraud, no other KYC services. Only fast and high-quality recognition, which, in our opinion, should have partially turned the digital world around.

Happened. In March 2015, we introduced the first version of the product – Smart PassportReader, which was “trumpeted” on all resources (which is only our article on Habré, where we talk in all colors about the features of the task and the ways of using technology in business). Although initially the market reacted sluggishly to our innovation, today only a few banks do not use this technology in mobile applications.

As the number of customers grew, so did the number of requests to add new features and capabilities. The base of supported document types was expanding, the list of supported languages, the number of used ones increased, the code base was bloated (despite the fact that over the past five years, some subsystems were rewritten several times “from scratch” so as not to drown in the future in support of legacy code). Of course, this all affected the final size of the SDK – the volume Smart IDReader in “full configuration” for iOS, it began to weigh more than 200 MB. A lot of? Yes, insanely many! However, a huge list of software capabilities was more evident than any megabytes.

Remote work, with its regularity and uniformity, gave us the opportunity to analyze our product in detail and communicate with customers how they use our SDK. It turned out that many customers use the functionality of recognizing payment cards and barcodes separately from passports (in fact, in different business cases and applications), which made us realize the need for a product for recognizing such objects. We named this product Smart Code Engine… Of course, we not only isolated the limited functionality from the previous SDK, formed a separate team of powerful guys who significantly expanded the engine’s capabilities in terms of recognizing codified objects. And most importantly, we have formed the key requirements for the product: work quickly, work under any conditions, work accurately.

As a result Smart Code Engine provides automatic reading of bank card data, machine readable zone and barcodes in a wide range of image capture conditions. The system is resistant to changes in lighting, shooting angle, distortions, angle of rotation, excessive and insufficient lighting. The system works in real time and is capable of recognizing bank cards and other objects on modern devices at a rate of 25 frames per second.

Smart Code Engine allows you to scan and recognize bank debit and credit cards of the “Mir”, Visa, Mastercard, American Express, JCB, MIR, Maestro, Unionpay and Diners Club payment systems issued by various countries of the world, providing extraction of not only numbers (12-19 digits), but also the period of validity and the name of the owner. Recognition of all types of bank cards is supported: with embossed, indent and flat printed, horizontal and vertical identification data, and those cards on which the data is located both on the front and on back side. In the new product, users can now recognize bank cards with an IBAN number, which are widespread in the EU countries. Barcode reader supports QR Code recognition (including various design versions), AZTEC, PDF 417, Data Matrix, CODABAR, CODE_39, CODE_93, CODE_128, EAN_8, EAN_13, ITF, UPC_A, UPC_E. The module for recognizing machine-readable document zones (MRZ), in addition to the international ISO / ICAO standard (IEC 7501-1 / ICAO Document 9303 ISO), takes into account local regulations (Russia, France, Switzerland, Bulgaria, Ecuador).

The second “rethinking” refers to our flagship solution – the ID-document recognition engine. Although at the very beginning we positioned our product to solve a whole range of tasks, from access control systems to complex artificial intelligence systems that autonomously serve people, the main application Smart IDReader found in the tasks of remote identification and identity authentication. Here are just a few “live” cases: registration of self-employed in the service “My Tax”, purchase and activation of a SIM card, registration in the Tinkoff mobile client.

Solving such problems requires more than just recognizing text details. This is how we created a new product Smart ID Engine, which, in addition to data recognition, provides multi-factor authentication due to the built-in face verification mechanism, analysis Liveness of the presented document, as well as highlighting signs of compromise.

In addition to analyzing individual text and graphic elements (including elements that change over time with changing environmental conditions, such as holograms), Smart ID Engine contains a collection of methods to prevent “physical” tampering with the integrity of the document (for example, detection of photo tampering).

Such a vector in the creation of a complex system of identification and authentication of a person opens up new frontiers and new sales markets for us, expanding the geography of the system’s applicability in every sense. And here we also decided to “shoot”: we made recognition of the Arabic script and languages of the Indo-Iranian group. For a moment, we are talking about 21 jurisdictions with a total population of 500 million people that do not use Latin inscriptions in national documents. Arabic recognition implemented for 73 types of documents, including passports, ID cards and driver’s licenses of the following countries: Algeria, Bahrain, State of Palestine, Egypt, Jordan, Iraq, Iran, Yemen, Qatar, Comoros, Kuwait, Lebanon, Libya, Mauritania, Morocco, UAE, Oman, Saudi Arabia, Syria, Sudan, Tunisia. In total, today we support 99 languages of the world, and started 5 years ago with two – Russian and English.

Finally, the last third “rethinking” of reality and the presentation of a new product, which was so quickly born during the acute phase of the pandemic – Smart Document Engine… This engine solves the problem of automatically extracting data from standard forms of documents, strict reporting forms, primary accounting, financial, tax, legal, notary and other documents. Just like all other recognition engines Smart Engines, strictly on the device, without using external high-performance servers. Plus many auxiliary functions, such as checking the presence of a signature and printing, handwriting recognition, control of the topological integrity of the document, etc.

Of course, skeptics can give up and say that the topic is completely meaningless and stupid, because who may need to recognize documents on mobile phones or tablets. I have an answer to this. Firstly, optimization of algorithms for weak computing architectures allows us to make a solution that works with unrivaled speed on workstations and servers (the processing time for 1 page of an A4 document on AMD Ryzen 7 3700X is about 2 seconds). And secondly, we have already met exactly such skeptical forecasts, 5 years ago, when we released a passport recognition product.

What’s next? Entering Western Markets? Sales development? PR and marketing and stuff blah blah blah? Of course not. In the West, as in the East, we already exist. Sales are growing. PR and marketing is already set up (since you read this article to the end). Therefore, let’s focus on the case:

adding new codified objects: recognition of individual text lines, recognition of VIN numbers, recognition of meter readings and others;
the addition of new languages and scripts, for example, more than 20 languages are used in India;
increasing the speed of recognition;
expanding the list of supported document types;
a lot of other interesting work.