Why does Moscow need its own driverless cars?
In 2019-2022, we had three separate and complementary projects. There was a lot of news about them in the public field, and many probably managed to get confused about where which drone went and how it all ended. Now let's put everything into pieces.
Administer it
Our first project is a car for administering paid parking based on Hyundai Solaris. The idea was to try a drone as a performer of routine work, automate it as much as possible, and in the future, refuse the services of a driver. Equipment of the car for unmanned driving (installation of drives and actuators, lidar, video cameras, inertial measurement system, etc., as well as the development and configuration of software for the car was carried out by our colleagues from the Moscow Automobile and Highway University (MADI), with whom we became friends back at the Up Great competition. From March 2020 to June 2022, the car drove almost every day on a fixed route in the Chistye Prudy area, and during this time it managed to drive more than 10 thousand kilometers around the area, learning to drive on its own.
There were some incidents: during the pilot, our car managed to become involved in a small incident – a car driving from behind drove into us. The driver was gaping, which happens to everyone. The news later wrote that there was the first insurance loss in history for an accident with an unmanned vehicle, settled under comprehensive insurance. And it is true. After minor repairs, the car was returned to the route.
Much more important is that at this stage we were able to test different scenarios for using V2X technologies (after all, an unmanned vehicle that cannot communicate with the infrastructure is just an expensive toy). Here is one example. Driving out on the route from the institute’s yard, the car “understood” that there was a gate in front of them, stopped, waited until it automatically opened and drove onto the road network. The gate, in turn, “understood” that a car had driven up to it via the RFID tag signal, and began to open on its own. The car understood that it couldn’t drive until the gate opened and the traffic light in which we installed the controller (information from which was transmitted to the car via LTE) turned green. It is clear that this is a fairly simple technology, but it was still one of the most important stages of drone training.
Also, at one of the intersections, my colleagues from MADI and I installed a camera to see what was happening around the blind turn. Objects in this space were invisible to onboard cameras and lidar. The stationary camera acted as spare eyes and informed the vehicle in advance of an approaching vehicle.
At the same time (which is extremely important) we learned how to make a digital twin of the road – a high-precision digital map of the city’s road network with traffic lights, markings, signs, turns and buildings. First, we surveyed the route using geodetic instruments. After this, the drone drove several times along the route under the control of the driver to record a cloud of all route points in 3D with lidar. Then, together with the MADI team, we combined the two resulting sets of data, simultaneously clearing the clouds of “garbage” (temporary objects that were on the route at that moment, for example, parked cars). The output was a digital twin. It looks about the same as a Yandex, 2GIS or Google map on your phone, but only with an accuracy of 2-3 cm.
To eliminate confusion, let’s clarify right away: in Moscow there are several projects that are called “digital twins.” Among them are digitized streets with transport infrastructure facilities; A 3D copy of the city territory, on which all networks of engineering and transport communications are mapped. All these tools are used to design and plan the development of territories and the road network.
It is clear that it is impossible for a pilot to predict all situations before launching, but that’s why he is a pilot. Quite quickly we realized that the digital twin of the route must be dynamically updated, especially in a city like Moscow. Let’s say that somewhere construction workers put up scaffolding (or utility services came to fix the accident), blocking a traffic lane. In the digital twin, created just a couple of days ago, there is no information about the overlap – the unmanned vehicle in this place will simply hit the wall – this is how the forests that suddenly appear on the way will be perceived. Ideally, before setting out on a route, the drone would receive all the necessary map updates, and even better, in real time.
A by-product of the camera-equipped vehicle's drives was data on available parking spaces along the streets. They were transmitted to our information system, but were not broadcast anywhere outside – after all, it was a pilot. But it became clear that drones can become assistants for collecting data, which can then be used in various useful services.
For greater clarity, a few words should be said about the legal side of the issue. When our driverless car was launched at the beginning of 2020, government decree No. 1415 “On conducting an experiment on trial operation of highly automated vehicles on public roads” was in force. This document allowed, under strictly defined conditions and in certain cities (including Moscow), to test unmanned vehicles on public roads. One of the key conditions is the presence of an operator in the driver’s seat, who can intervene in the control if necessary. From the outside, of course, it didn’t look very “unmanned”, but believe me, our car drove most of the time on its own, and the operator carefully watched the road, but rarely interfered with the controls. In addition, each drone had to undergo mandatory certification, and changes to the design had to be included in the documents.
Exactly at the address
The second project is a drone that drove through the closed area of the City Clinical Hospital No. 1 named after N.I. Pirogov within the framework of a joint project of MosTransProject, MADI and the Moscow Innovation Agency. The idea for such a project was born in the midst of a pandemic, in April 2020. Nurses then rushed around the vast territory of the hospital, delivering tests to buildings. They walked an average of 10 thousand steps every day: the machine could have greatly relieved the medical staff, who obviously had something to do during the pandemic. In addition, we had no experience organizing logistics in closed areas using drones, and we had a great chance to get it.
By July 2020, we had passed all the approvals, and in August we made a digital twin of the hospital territory (approximately the same algorithm as with the car at Chistye Prudy). It must be said that the traffic organization there was not the most optimal, but a drone requires clear markings, signs, parking spaces and routing. Together we fixed everything and it became more comfortable to drive.
Since September 2020, a Ford Focus from MADI’s unmanned fleet has been driving around the territory, and by the end of the year they equipped a brand new Lada Vesta Cross and improved the software. The innovation of our colleagues was an algorithm for avoiding randomly walking pedestrians, because part of the route passes through a pedestrian zone, where hospital staff and visitors not only crossed the roadway, but also walked along the road, trying in every possible way to be in the path of a car. The algorithm predicted the trajectory of pedestrians, calculated vehicle traffic corridors, and determined safe detour routes.
In total, the vehicle made eight trips during the day: four to collect materials and transport them to the laboratory, and another four to move empty containers back into the buildings. The nurses found out that the car had arrived and was waiting for loading through a telegram bot. After the tests were placed in the car, the health worker pressed a button on the roof of the car, and it drove away on its own – the driver-operator did not interfere with this process. Ambulances often drove through the area; the vehicle itself recognized the obstacle and let it pass.
As part of this pilot, we again came to the conclusion that the digital twin of the road must be constantly updated. On the territory of the hospital, for example, reconstruction of the building with fencing of the territory can begin. Or, for example, the same snowdrifts after heavy snowfall or a pile of autumn leaves that have not yet been removed. For the “eyes” of a self-driving car, this is a piece of wall that it knows nothing about. This could lead her into a dead end.
In the ideal world we will one day arrive at, all city streets will have a single, constantly updated digital twin to which any drone can connect. The source of data enrichment for it can be other drones or sensors installed on public transport and municipal vehicles.
What happened next?
On July 1, 2022, Decree No. 1415 ended and both of our projects were completed. By this time, the law on experimental legal regimes had already been adopted and entered into force, which made it possible, among other things, to continue testing unmanned vehicles. Many companies (Yandex, Sber) then acquired their own cars, which they wanted to test in Moscow. The transport complex, in this regard, received many proposals, so it was decided to go the full route of creating our own unmanned vehicle from scratch, entirely on our own, in order to understand all the technical nuances from the inside out. This work was entrusted to us, which is logical, since MosTransProject is, among other things, a research center.
We have done serious research work. We studied global and domestic experience, what the machines of leading companies consist of, the software they use, and determined for ourselves what set of equipment to buy. Since the previous two pilots, hardware and software have made a qualitative leap, so our new device had to become more technologically advanced. We installed a lidar, six radars and three cameras in the new car, two of which worked as a stereo pair (previously, for example, there was a lidar and only two cameras). The task was also complicated by the large-scale rise in prices of semiconductors, which occurred precisely in 2021. But in the end we managed it successfully, although it took a lot of time.
We again took the Lada Vesta as a “guinea pig” because we needed a modern car with an electronic gas pedal and electric power steering, where we could install controllers for remote control. Next, it was necessary to solve the issue with the brakes, which cannot be controlled via the controller. At first, we had the idea to buy a controlled valve body from a Toyota Prius, the use of which we already have in other drone projects. But we refused this, since it would be considered gross interference with the braking system, which would entail re-certification of the car. Then we found an alternative solution – a remotely controlled servo drive, which we installed on the brake pedal and automatic transmission selector. By the end of 2022, we managed to get the car running, but we tested, tuned and tested it in closed areas, including in the courtyard of our institute. We are now using the car for scientific purposes to test various hypotheses related to autonomous vehicles. We are already using the experience gained over four years together with colleagues in the transport complex in other projects.
What conclusions have we reached? Technically, creating an unmanned car is not difficult – software, sensors, cameras, lidars and other equipment – everything is available, order and buy. The hardware is relatively inexpensive. You can install all this on a car and make it move independently with the efforts of a small team – in fact, we had several people doing this. This does not require any extra budgets; the most expensive, perhaps, is the machine itself and the lidar.
The most difficult stage is to train an unmanned vehicle to respond to emergency situations that cannot be foreseen in a digital twin of the road (even dynamically updated). Many foreign companies are still faced with this and have not found a solution. As an example, I will cite last year’s case in California, where an unmanned taxi of the Cruise service did not have time to quickly react to a pedestrian who suddenly appeared on the road on the road, and hit him. Thank God everyone is alive. The company put the driver back behind the wheel as a safety net and said that it would think about how to improve the operation of the software. Another example comes from 2022, when in San Francisco a dozen self-driving taxis created a traffic jam at an intersection at night due to a software glitch.
The digital twin of the road is, rather, an ideal city with ideal walls and intersections. But in real life on the streets there are always elements of chaos and Brownian motion, sudden pedestrians, animals that the drone must, if not predict, then at least see in time and make an instant decision. Only in this case can the car be trusted with the lives of passengers. Yes, we can implement a behavior algorithm (for example, to avoid obstacles), but for full training we need to drive a lot with the driver behind the wheel, collecting and analyzing terabytes of data. This is exactly what requires a lot of resources, including temporary ones. This explains the huge amounts of money that the largest corporations are investing in unmanned projects. In fact, no one in the world has finished this work yet.
We will definitely talk about our future projects in the field of unmanned vehicles in one of the following articles!
Team of unmanned technologies “MosTransProject” – Yuri ButenkoVladimir Bondarenko, Anastasia Tovmasyan, Roman Khikhin, Fyodor Utkin, Nikita Novikov, Alexander Belov.
