This robotic arm, controlled by thought, can turn, pick up objects, and even feel them.

Nathan Copeland learned to control the robot arm with his mind, but his movements were slow. The developers have now implemented haptic feedback.

Nathan Copeland was only 18 years old when he was in a car accident and was paralyzed. The misfortune happened in 2004. Since then, Nathan is practically unable to move and has lost the sensitivity of most of his body. True, he is not 100% paralyzed – the sensitivity of the wrists and several fingers is preserved. He can also move his shoulders a little. At the hospital, he became one of the participants in a pilot study.

A team at the University of Pittsburgh needed a volunteer to practice the mental control system for a robotic arm. The task was simple: to learn how to control the robotic arm, simply by thinking about different movements. All this is one of the works on creating a neural interface that allows you to control mechanisms or software with the help of thought. Companies like Kernel or Elon Musk’s Neuralink are doing similar work.

Copeland was fortunate enough to be a participant in the experiment, since the criteria for selecting volunteers are usually extremely stringent. The person must have some type of injury, plus the volunteer must live relatively close to the laboratory.

Nathan met all the requirements, so he was chosen. Soon, the newcomer underwent an operation, during which neurointerface electrodes were connected to his motor (motor) cortex and somatosensory cortex. With the help of the latter, scientists read out electrical signals from the brain, mainly those that arose at the thought of the intention to move the fingers or wrist. These impulses were transformed by a computer system and transmitted to the manipulator control interface. After three sessions of work, Copeland learned to work with the system. In particular, he was able to move balls and grab cubes off the table just by thinking about it.

But that was just the beginning. In research, published the other day Science reported, in general, a sensational thing: now the volunteer feels the touch of objects. What kind of these sensations are not reported, and it is difficult to convey such things in words. But the fact that the touch of a robotic manipulator to objects feels like touching the same objects with one’s own fingers is a fact.

According to the developers, the speed of reaction and movements of the robotic arm has accelerated – before the movements were slowed down. The volunteer had to look at what the manipulator was doing and think over the actions, correct them. Now, the process has been added to the sense of touch, which allows you to act reflexively. In some cases, the speed of movements of the manipulator doubled after adding a tactile response. This development, without exaggeration, can be called a breakthrough in the issue of restoring legal capacity to paralyzed people.

According to the research team, touch is an important factor in regaining mobility. When a person feels touch, he is able to act faster. Visual observation of the process is less effective than tactile sensations and eye contact. Moreover, if there is touch, there may be no eye contact at all. When an ordinary person interacts with an object, the actions are mainly dependent on sensations.

The human brain receives information, processes it and sends signals further through the body. The usual action, like grasping the cup with your fingers, is not so simple. During the movement of the fingers, the brain corrects the actions of the hand. When fingers touch the cup, the brain receives tactile feedback and guides the hand for maximum grip.

Since the volunteer’s brain was intact, in theory it could maintain “tactile contact” with the fingers. Unfortunately, most of the electrical signals in the participant’s nervous system did not travel from the brain to the limbs. Accordingly, the researchers had to develop a system that simulates tactile sensations. The main task was to make the sensations real and coincide with the actions of the manipulator. So, when the robotic little finger touches an object, Copeland should have felt it with his own little finger.

To provide feedback, Copeland was implanted with several more electrodes. Two sets of electrodes are responsible for analyzing brain signals when thinking about performing an action with a hand, and two more – simulate sensory sensations. In fact, a small electrical signal in the electrode can provide the sensation of touch. It was necessary to synchronize the generation of such signals with the movements of the manipulator.

To create such a system, the researchers took advantage of the fact that Copeland retained sensitivity in the thumb, index, and middle fingers of his right hand. The researchers touched them with a cotton swab and analyzed the ECG signals, as well as the areas of the brain where these signals occur. As a result, we managed to understand which part of the brain is responsible for which finger. After that, the scientists generated similar signals and applied them to the electrodes. According to the volunteer, he feels like something is touching the base of his fingers in the upper part of his right palm. It might be pressure, warmth, or just an unusual tingling sensation, but not pain.

Before the introduction of the tactile system, Copeland did a good job of gripping and moving objects. It took a couple of years to train, but the results were excellent: while an average person takes about 5 seconds to perform such an operation, Copeland took 5 to 20 seconds. Further trainings did not give any result: the volunteer, as it were, reached a plateau, without deterioration or improvement in results.

Everything changed after the addition of the tactile system. As mentioned above, the result was immediately improved by 2 times or more. Of course, I had to train again, given that Copeland’s tactile sensations are not exactly the same as that of a healthy person. He himself says that he began to act much more confidently even during the first test, because he did not have to completely rely on vision.

If there is no feedback, you have to rely only on vision. In this case, the person must first make sure that the object is indeed captured. If yes, then you need constant visual control during its movement.

Creating realistic sensory cues is a big win, experts say. This means that scientists are getting closer to the task of imitating the natural movements of a healthy person. It is important that the action takes place without too much delay. Tactile feedback is exactly what helps to equalize the time it takes for an ordinary person to perform an action and a person with damage to the nervous system.

For a tactile system, the signal processing time, lag, is about 30 ms. The visual system, on the other hand, is from 100 to 300 ms. Imagine trying to grab a slippery cup. If you use only your eyes, you will know that you dropped it only after you see the falling cup.

Despite all the successes of the developers, the system is still at the stage of “laboratory prototype”. Copeland can only work with the manipulator in the laboratory. He cannot carry it or take it home with him. True, Copeland has a simplified system designed so that he can control his personal PC. The system is advanced enough to be able to play Sega Genesis games on an emulator. Copeland was even able to draw a cat and turn it into an NFT.

Another problem with the system is that it requires a wired connection. But this task seems to be easier to solve than others – many companies have developments of wireless neural interfaces.

One last thing: Copeland’s tactile sensations when touching objects with a robotic hand do not always correspond to natural ones. Therefore, it is still more difficult for him to control the capture / movement of objects than for an ordinary person.

But, in any case, the progress is huge. We can already say with confidence that the human brain is still attached to his flesh, but is no longer bound by it.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *