CIMON-2: (un) Doomsday, or how IBM Watson climbed above the clouds
CIMON – Crew Interactive Mobile companioN (hereinafter referred to as "Simon") is a scientific development of the aerospace agency Airbus and IBM, sponsored by the German Aviation and Space Center DLR. Last year he was first tested “in battle” on the ISS, and this month they began testing the second version of the assistant. Since we are interested in everything related to high-tech and breakthrough communications – for example, we already wrote about working with IBM Watson – we could not get past this news and prepared a translation-compilation of several foreign materials to understand for ourselves and tell you – Why do astronauts conditional Alice / Alexa / {you name it}? Under the cut, you will find a chronology of events, quotes from those involved and a small conclusion. Enjoy reading!
CIMON, November 2018
In November 2018, Simon successfully passed the first tests on the ISS in the Columbus module – for one and a half hours, the German astronaut Alexander Gerst worked with him.
Gerst has been working at the station since June (Horizons, 56 and 57 missions to the ISS), and his tasks included testing Simon. This 5 kg plastic sphere, which was printed on a 3D printer, is the first AI assistant for astronauts created to study the interaction of man and machine in space.
November 15, 2018, University of Lucerne, ground control center. Simon’s team tensed noticeably: two and a half years of intensive development and training passed, and now everyone was waiting for the tests to begin. Having waited for Simon to download all the updates, as well as test the sound and navigation camera, Alexander Gerst activated and “met” his new colleague. The first working meeting of the astronaut and the "soaring in the air" assistant lasted 90 minutes.
Simon’s autonomous navigation worked successfully – he not only hung at the right point in space, but also turned in the given directions. He was also able to find Gerst's face to make eye contact. As a demonstration of precisely assistant skills, Simon showed instructions for a school experiment on crystallization, then a video on assembling a Rubik's cube and lost a musical fragment (Kraftwerk – The Man Machine). Then Simon tested ultrasonic sensors, similar to those used in parking sensors, and made a video recording of the astronaut using the built-in cameras. At the end of the “meeting” Gerst fixed the assistant on a tripod.
A fragment of testing can be seen here – Kraftwerk and a couple of fun moments are attached:
Total:
the assistant uses air currents to move and fix the position;
cameras are used for face recognition and photo and video recordings;
ultrasonic sensors coupled with cameras help to avoid obstacles;
some of the built-in microphones also work for spatial orientation, and directional microphones provide speech recognition;
the assistant has two AIs: an “on-board” one from Airbus for orientation, a “network” IBM Watson for speech analysis.
“Simon is a huge step for space travel, which laid the foundations for creating assistants for extreme conditions,” says Airbus project manager Till Eisenberg. Dr. Christian Carrash, DLR project manager, echoes it: “Simon hears, sees, understands and speaks – it’s incredible to realize that. The history of aerospace is literally being written before our very eyes, and this, I hope, is the beginning of longer experiments in this direction on the ISS. I admire the interaction with AI. Simon is the only system of its kind because it was created exclusively for the ISS. We stepped into uncharted territory and expanded the boundaries of technological competence in Germany. ”
The assistant uses on-board Wi-Fi to further transmit data via satellite to Earth and to maintain a connection to the IBM cloud. Matthias Biniok of IBM explains the chain of action: “When Simon is asked a question, the audio is first converted to text, which is then interpreted. IBM Watson not only understands the content and its context, but also the intention behind the statement. As a result, a text response is formed, converted into audio and sent back to the ISS to get a natural, dynamic communication. ”
Bernt Rattenbacher, team leader of the ground control center at the University of Lucerne: “Data to Earth is transmitted via satellite to NASA / ESA and the Columbus ground center in Oberpfaffenhofen. From here, the signal is transmitted to our center, which is connected to the IBM cloud. The time to deliver a signal to each side is 0.4 seconds. To ensure data security, we use many firewalls and VPNs. ”
Airbus believes that thanks to Simon, the crew will be able to do more – on the example of actions on check lists. It will be possible to involve an assistant in the process, so that he helps make the routine more effective, which will contribute to the success of the mission and potentially increase its safety (the assistant can be used as an early warning system about problems). Finally, an assistant can have a beneficial psychological effect on astronauts during a long stay in a small group – and it is through this that crew members of any space mission go.
CIMON-2, December 2019
The first version of Simon spent 14 months on the ISS and returned to Earth on August 27, 2019, and on December 5 the second version was delivered to the ISS, in the same Columbus module. The physical characteristics of Simon-2 are similar to its predecessor – the same 5 kg of weight and the size of a soccer ball. The functionality is formally the same, but in fact – updated. Emotional intelligence based on IBM Tone Analyzer was added to the new version to make not just a scientific assistant, but also an empathic interlocutor.
In addition to pumped emotional intelligence, Simon was updated with iron, put more sensitive microphones, improved spatial orientation and increased software stability. Airbus has also updated the assistant’s navigation so that it can move more smoothly and autonomously around the ISS. The second version of the assistant will stay on board for up to three years, during which astronauts will track how the assistant’s emotional analysis works.
“The first version showed that it understood not only the essence of voice messages, but the intentions of crew members,” says Matthias Biniok of IBM. – Simon-2 goes further: now he can evaluate the emotions of the crew and react to the situation in an appropriate manner. Simply put, Simon-2 in real time analyzes whether his interlocutor is satisfied, or whether something is disturbing / angry, and so on. ”According to Biniok, this will help Simon become a counterbalance to“ group thinking ”when the opinions of people working for a long time in a group, gradually come to a consensus or become very similar. It is expected that Simon will be able to determine such a situation and react, either putting forward an objective, neutral point of view, or even become a “devil’s advocate”, that is, take the opposite point of view of the crew.
Nevertheless, the role of a negotiator is rather a future goal. How can Simon-2 be useful now? Biniok continues: “Time is an especially expensive and limited resource on the ISS, so if we can save the time of the astronauts by helping them with task planning, it will be very valuable. Simon also supports experiments – imagine that you are an astronaut and you have to conduct a series of complex studies for which you have been given a lot of documentation. And if you lack some information or are simply puzzled by the current step of the experiment, you are forced to open the documentation for clarification – it will take some time. Instead, you can ask Simon a question, for example, “what's the next step, Simon?” Or “why is Teflon used here and not other material?” In addition to working with existing documentation, an assistant can create it himself, recording what is happening with the help of cameras and microphones. And he can do it autonomously – in theory, an astronaut can tell an assistant to visit a location, take a photo, return to the astronaut and show the result.
Output
Honestly – some of the above statements look, if not utopian, then very, very bold. The speed with which Simon communicates is still small; empathy also raises questions. There is a suspicion that at the current stage such an assistant can hardly optimize any processes and gain time for the crew – it is obvious that while this is far from Jarvis from the movie or Cortana from one well-known game, but something else is important here – the beginning has been laid. If controlled robots have long been used both in space and in extreme terrestrial situations, then there were simply no autonomous and "tangible" assistants in such conditions before. It is hoped that the financing of the project will not stop suddenly and in the foreseeable future these assistants will become an important tool in space missions. The main thing is that everything will turn out better than with android David in the Ridley Scott film franchise … We closely monitor the development of events 🙂
Original articles
Conventionally, all articles about the first version of the assistant are about the same thing, but still some points are unique. The same with the materials on Simon-2, so we could not choose one article for translation, but we took a little bit everywhere.
