This article was prepared by me, my colleague @geoink and our leader Artyom Nikolaevich Volkov. You can also look at our previous work on this topic.
The work was defended and presented by us at the international conference ICACNGC 2023.
Analysis of promising 2030 networks such as 6G and telepresence services suggests that the future involves higher data speeds, low latency and greater availability of virtual and augmented reality. 6G networks are expected to significantly increase capacity and the ability to transmit data at much higher frequencies, which will open new horizons for user interaction with the virtual environment.
However, with the increasing capabilities of 6G networks and telepresence services, important scientific and technical aspects arise, including the issue of the “telepresence suit”. The telepresence suit is a technologically sophisticated device that is expected to be used extensively in such an environment.
Research should focus on developing telepresence suits that best meet the user’s performance and comfort requirements. This includes ergonomic design to eliminate discomfort and fatigue, optimizing power consumption and computing resources to ensure long-term device performance, and developing data security and privacy mechanisms to account for the collection of personal information in the context of telepresence.
In addition, it is important to focus on standardization and compliance to ensure telepresence suits are compatible with 6G network infrastructure and comply with communications and data protection regulations.
Therefore, research and engineering developments in the field of “telepresence suit” are essential to ensure the successful integration of telepresence services in the future 2030/6G networks and to ensure that user needs are met in this context.
Analysis of current solutions
Brain-controlled telepresence robot by motor-disabled people.
Research into brain-machine interfaces makes it possible to control robots and prosthetics using signals from the brain. This is useful for people with limited mobility. The basis is the recording of brain activity using neuroelectroencephalography (EEG). Collaborative control between the brain and the robot plays a key role in ensuring intuitiveness and efficiency. Interestingly, even patients located far from the control site can achieve similar results as healthy subjects.
This article presents the first results of the use of brain-controlled telepresence of a robot by users with disabilities. Patients located in a clinic more than 100 km away successfully controlled the robot despite the unfamiliar environment. The study found that shared control reduced users’ cognitive load and increased their attention span. Fast decision making proved to be critical, and users had to be quick and accurate in communicating mental commands for effective control.
The study highlights the importance of telepresence for people with limited mobility. This allows them not only to control robots in remote locations, but also to feel part of the environment, which is of great importance for the social and psychological well-being of this category of users.
Telemedicine and telepresence for trauma and emergency management.
Teletrauma programs allow rural patients to access advanced trauma and outpatient medical services that are often limited to urban areas.
The study highlights the positive contribution of teletraumatology to medical practice as it supports initial assessment, treatment and care of patients, improves patient outcomes and provides cost savings. It is important to note that 6 patients were potentially life-saving through telemedicine consultations and that such programs avoided patient transfers and therefore saved costs and resources. This demonstrates the significant potential of teletraumatology to improve access and efficiency of health care in rural areas.
This article demonstrates the potential life-saving role of telepresence in health care delivery, especially in the context of remote or hard-to-reach regions where access to high-quality health care facilities is lacking.
Long-Term Evaluation of a Telepresence Robot for the Elderly: Methodology and Ecological Case Study
The authors explore the feasibility of using telepresence to support social activity in older adults and evaluate the attitudes and acceptability of such systems for users sharing space with a robot. The main objective is to develop a methodology for assessing the long-term experience of using telepresence and its impact on users in real-life conditions of everyday life.
This article presents the MARTA (Multidimensional Assessment of Telepresence RoboT for older Adults) methodology, designed for a long-term study of the adaptability and compatibility of robotic telepresence in supporting social interaction among older adults. The methodology involves monitoring a number of relevant variables over time and also emphasizes developing a protocol for deploying and maintaining the technology to minimize potential problems associated with technical failures. The study was evaluated in an ecological long-term case study using the telepresence of a Giraff robot in the home of an elderly couple. The case study results confirm the effectiveness of the proposed methodology and support the idea of using telepresence robots to support social communication and presence.
Telepresence also has the potential to help older adults continue to socialize in a fast-paced world. This technology allows older adults to remain connected to family, friends and community despite physical limitations or distance. This helps older people maintain an active and rich social life, which can have a positive impact on their overall well-being and quality of life.
Telepresence robot system for English tutoring
One of the key issues addressed in this article is the lack of native English speakers in schools in Korea. The high demand for English education has led to many parents investing in improving their children’s English language skills. However, the number of native English speakers is not enough to meet the needs in schools. This leads to a shortage of teachers, especially in suburban areas. Telepresence via video communication system is seen as one of the solutions to this problem, allowing lessons to be conducted online where teacher and student can communicate regardless of their location. However, existing telepresence systems have limitations associated with fixing the position of the device, which limits the freedom of movement of the teacher and his ability to interact with students. Telepresence robots may offer an alternative as they can be controlled remotely by teachers and provide more active and attentive interaction with students
From the English lessons conducted using the robot’s telepresence system, the following key points were highlighted: Teachers were satisfied with the opportunity to conduct lessons without moving to the classroom, as well as active interaction with children using the robot. Despite the advantages, teachers also identified disadvantages, including limited visibility and difficulty controlling the robot, especially when the collision avoidance system intervened. The students showed interest in the robot and actively participated in the lessons, but there were problems with the low concentration of other children when the teacher was interacting with one of them, due to the narrow viewing angle of the monitor on the robot. Children also had difficulty viewing educational materials due to the TV being located far away from the robot, which made it difficult for them to concentrate on the lesson. The results of a survey of children showed an increase in interest, confidence and motivation in learning English after lessons conducted using a telepresence robot.
Thus, the robot’s telepresence system has shown its potential in teaching English, however, there are some limitations that require improvement for a more effective educational process
In today’s world, embedded systems play a key role in many aspects of daily life, providing automation, monitoring and data collection. In the context of evolving Internet of Things (IoT) technologies, communication between different devices is becoming an increasingly important aspect. Several protocols and standards exist to enable this communication, each with its own characteristics and applicability.
This article focuses on the interaction between Raspberry Pi 4 Model B and WEMOS WiFi & Bluetooth ESP32. Of particular interest are three main communication protocols: SPI (Serial Peripheral Interface), Bluetooth and I2C (Inter-Integrated Circuit). These protocols provide efficient data transfer and control between devices, while having their own unique characteristics designed for different application scenarios.
The study takes a closer look at each of these protocols, analyzing their data transfer delays. This research is being conducted as part of the development of a telepresence suit. In the following sections of the article, we will take a closer look at each of the three protocols and present the results of our research, allowing readers to gain a deep understanding of the interaction between the Raspberry Pi 4 Model B and the WEMOS WiFi & Bluetooth ESP32.
The article also presents the architecture of the on-board network of a telepresence suit. User scenarios are presented, and a stack of scientific and applied problems is defined, the solution of which will bring the implementation of telepresence services closer.
1. General architecture
The telepresence suit is a peripheral device located at the user level. This layer is the starting point for data transfer. After this, a connection is established with the router, which in turn is connected to the edge server and the server’s multi-controller. The edge server and multi-server controller communicate with each other to ensure seamless communication. The edge server is then connected to the router and router. The router provides communication with the robot, allowing it to be controlled and data transferred. The router, in turn, is connected to cloud storage, which allows you to save and retrieve information from the cloud. This complex networking structure enables the functionality of the telepresence suit and its interaction with the robot, as well as with cloud data storage, to ensure reliable and efficient operation of the system.
2. Example of use
The telepresence suit reveals its potential when integrated with a humanoid robot and hybrid drones, providing unique capabilities for solving critical problems in conditions of destroyed infocommunication infrastructure. In this scenario, the suit plays a key role in controlling the humanoid robot, which functions as a mobile base station. This humanoid robot mobile base station collaborates with hybrid drones to create a mesh network. A mesh network is a distributed and self-organizing infrastructure where each device in the network can serve as both a receiver and a transmitter of data. This provides reliable communications and extensive coverage in environments where traditional communication channels may be interrupted. This joint system of humanoid robot, drones and telepresence suit is of utmost importance in emergency situations such as natural disasters. It facilitates the search and rescue of people trapped under rubble or lost in difficult conditions, providing unique capabilities for detection and coordination of actions. This system is an outstanding example of innovative application of technology in the field of emergency and rescue operations.
3. Telepresence suit architecture
This system uses the computing resources of a Raspberry Pi 4 Model B in combination with five ESP32 microcontrollers that communicate via the SPI (Serial Peripheral Interface) interface. Each of the ESP32 modules performs the function of collecting data from sensor devices, providing environmental monitoring. After collecting data, the ESP32 transmits this information to the Raspberry Pi 4 Model B. The Raspberry Pi 4 Model B, in turn, acts as a data aggregator and transmitter. The collected data is processed and transmitted over the network using the MQTT (Message Queuing Telemetry Transport) protocol through a router. MQTT Broker accepts and manages this data, providing an interface for multiple devices that can subscribe and receive up-to-date information. Such a system provides processing and transmission of data from sensors to the central infrastructure, including environmental monitoring, process automation and remote control based on the collected data.
Experiment and result
As part of this research, ready-to-programmable ESP microcontrollers and the computing resources of the Raspberry Pi 4 Model B were used to create an on-board telepresence suit system. This choice of tooling, although accompanied by some additional delays in implementation, did not contribute to the overall trend identified in the study.
Using five ESP32 microcontrollers with a Raspberry Pi 4 Model B provided an opportunity to test the viability of the proposed architecture and explore in detail the non-functional limitations of the SPI interface. This is an important aspect that allowed us to evaluate the performance and reliability of the system.
Establishing a connection using SPI technology
From the presented graph we can conclude that the best quality of data transfer is observed within 110 bytes. Exceeding this value results in a significant increase in data transfer latency to the Raspberry Pi 4 Model B, which becomes noticeable to human perception. On the other hand, the WEMOS WiFi & Bluetooth ESP 32 device shows no noticeable latency even at higher data volumes.
Establishing a Bluetooth connection
Based on the analysis of the graph, we can conclude that the use of Bluetooth technology is not optimal for data transmission purposes in a telepresence structure, since the observed delays exceed acceptable parameters.
Establishing a connection using I2C technology
Acceptable delays are within data transfer limits of up to 10 bytes. However, exceeding this value leads to unacceptably long delays, which does not meet our requirements. Also, when transferring data higher than 32 bytes, a failure occurs in the software part.
By analyzing the presented graphs, we can conclude that the SPI protocol is optimal in data transmission in terms of speed. Despite the occasional artifacts, their number is limited, which allows us to continue working with this protocol. In the context of creating a telepresence suit, which requires interaction with five WEMOS WiFi & Bluetooth ESP 32 devices, it is necessary to evaluate the delays for this particular number of devices
Connecting five WEMOS WiFi & Bluetooth ESP32 and Raspberry Pi 4 Model B
The detected delays are characterized by significant values, which have a significant impact on the efficiency of the system.
Ultimately, we came to the conclusion that we needed to create our own data transfer protocol. Future development plans include improving the software component of the system. The main goal is to reduce current latency levels to 1 millisecond to ensure higher efficiency and responsiveness of the system.