What touch virtual walls?

Over the past few years, devices that allow the user to plunge into virtual reality (VR) have become much more complex and better. VR is used in education, and in art, and in entertainment, and even in medicine (no, I’m not talking about Surgeon Simulator). Thanks to VR, a person can see something that he would never have seen in real reality, such a pun. However, for a complete immersion in the world of dreams, one of the most important sensations, which provides a person’s contact with the outside world, is lacking – touch. We constantly feel something to the touch: keyboard keys, doorknob, tea cup, etc. In VR, all objects, no matter how realistic they seem to the eyes, are devoid of any physical representation in reality. A group of scientists from Carnegie Mellon University (USA) decided to fix this using a fairly simple device in concept. What the invention consists of and how it works, we learn from the report of scientists. Go.

Study basis

Modern VR systems use controllers (and even ordinary gamepads from game consoles) to track the position of the user’s hands, control the actions of their virtual avatar and for vibration-tactile response.

Those of us who have a game console know that gamepads can vibrate at certain points in the game (battle with the boss, collision with an opponent in races, etc.). VR also uses this method of communication with the user, however, it does not give that fullness of sensations that would give a touch.

If we talk about the walls in VR, then they successfully implement the concept of the Schrödinger cat – they seem to be there, but at the same time they are not. The user can go through them (in whole or in part) intentionally or not. And this greatly affects the feeling of the world created in VR.

To get rid of the above disadvantages is quite real, although difficult in terms of design and implementation. There are several options for creating tactile contact with VR objects, from the exoskeleton that organizes the user’s movements, to the physical restriction of the VR system in real space (i.e. real walls). However, both of these methods limit user freedom of movement, and therefore cannot provide a complete and comfortable immersion in the virtual world.

Image # 1: The concept of the Wireality system.

In the study that we are considering today, scientists decided to create their own version of the tactile contact system by means of threads attached to sensors on the user’s hands, which looks like Pinocchio. Such a system is autonomous and very mobile, not to mention its low cost – up to $ 50 apiece for mass production. The inventors named their creations Wireality, combining wire (wire, thread) and reality (reality) in one word.

Wireality system implementation

So, the main physical properties of the system that its creators wanted to get were ease of use (the system will be attached to the user), energy efficiency (an autonomous power supply is needed) and relatively low cost.

The conceptual and mechanical basis of the system are threads that will in a certain way stop certain sections of the user’s brush at the right time. As a material for the threads, it was decided to use fairly strong and thin steel cables with a nylon coating.

Image No. 2: prototype of the Wireality Thread Tagging Module.

When the threads are the limiters of the position of the hand, they should not be allowed to sag or to excess tension, because this will disrupt the system.

To prevent this from happening, it was necessary to use something that would effectively tighten the threads. The first options for the tightening module, as the scientists themselves admit, turned out to be cumbersome due to the engines, and the problem of heat removal appeared (the scheme above).

Image # 3: Final version of Wireality Thread Tagging Module.

Instead, flat springs were used, as in measuring tape. This option is much easier, cheaper, stronger and more compact. Even though the flat springs provide an exceptionally fixed traction, it was found that a retractor module with a traction force of 80 g perfectly eliminates the problem of sagging threads.

The threads are selected, the pulling module, too, it remains to determine the module for fixing the position of the brush. One of the most important factors in obtaining the most realistic sensation is the delay in the response of the fixation module to user movements. Wireality systems utilized cam and ratchet locking mechanisms.

The final design of the locking module consists of an acrylic ratchet mechanism with ratchet (gear with sharp teeth, roughly speaking) with a resolution of 8 ° per tooth and a tightening module in which the spring, spool and thread are installed.

Since the thread is wound on a spool with a smaller diameter than the ratchet, each tooth is equal to 0.84 mm “string travel”. The cam, in turn, attaches to the push pull DC solenoid (12 V).

Therefore, when the user’s hand is in contact with the virtual object, the system program starts the solenoid, pushing the latch into the ratchet, which blocks the coil from further rotation, i.e. prevents further release of threads, which stops the position of the hand. And due to the effort exerted on the thread by the user himself, the ratchet holds the latch, so the solenoid can be turned off quickly enough, which reduces energy consumption.

Image # 4: thread lock module.

One blocking module is responsible for one section of the user’s hand (for example, for the index finger), because they need several. Compact size (4.1x7x1.4 cm) and lightness make it possible to connect them into clusters (photo below).

Image # 5: cluster of lock modules.

Each module is housed inside a case of nylon and carbon fiber printed on a 3D printer.

The electronic component of the interlocking modules consists of the HBridge L298N dual motor driver controlled by the Teensy 3.2 microcontroller, which receives commands from the VR system via regular USB.

These electronic components are compact and practically do not consume energy when the solenoids are not involved. The dimensions and cost of the module can be reduced by using a transistor, which will be considered in further studies.

When Teensy instructs to block one of the threads, it activates the corresponding solenoid for 40 ms, which is the shortest interval for reliable blocking.

Having gathered all the modules together, scientists thought about where all this joy would be attached to the human body. The prototype, of course, is not yet ideal, and its size can be reduced, but even at this stage it is quite convenient to wear it. It was decided to place the Wireality system on the user’s shoulder by means of a special vest that distributes the load on the body (image No. 1).

As for the hands, it was necessary to understand the optimal number of contact “sensors”, which directly determines the number of blocking modules and, therefore, the number of threads used. Scientists have found that with a large number of contact points (for example, the tip of the finger, metacarpophalangeal and interphalangeal joints), more complex geometric shapes can be modeled. However, this inevitably leads to an increase in the dimensions of the device and its weight (a total of 15 modules are needed: three for the index, middle, ring and little fingers, 2 on the thumb and 1 on the wrist).

During the experiments it was found that such a large number of sensors is not necessary to obtain realistic tactile contact with a virtual object. Therefore, in the end, the number of strings and modules was reduced to seven: 5 for each finger, 1 on the palm and 1 on the wrist.

Sensors with threads are attached to the user’s hand through thimbles printed on a 3D printer and Velcro straps, which helps to distribute concentrated pressure.

To hold the user’s fingers at points in three-dimensional space to simulate complex virtual objects, precise tracking of multiple joints of the arm is required. In early prototypes (one of the options in image No. 2), a potentiometer was used, which made it possible to accurately track the distance to a point from the module, but not the azimuth or altitude.

Ultimately, Leap Motion, attached to the front of the VR headset, was used to track hands. Leap Motion provides data on the position of all joints of the arm in three-dimensional space. It is curious that the thimbles that are worn on the user’s fingers were made of a material that does not interfere with the operation of Leap Motion, since in the infrared they look the same as leather. Another important detail was the threads themselves, which were supposed to remain invisible to the camera, because their thickness was also not chosen by chance.

Oculus Rift was used as a VR system, and all virtual experimental sites were created using Unity. To detect contact with virtual objects, an ObjectCollider was assigned to each object or obstacle on the site (wall). When a contact occurs, an OnTriggerEnter event is dispatched. The event handler sends a lock command to the driver board via USB for the corresponding joint, i.e. the corresponding lock module.

Contact detection is performed in parallel for all joints of the fingers and hand, and the actuation of the solenoids does not interfere with each other, providing completely independent control, which is necessary for tactile visualization of complex geometries.

Video that describes how the Wireality system works.

Wireality System Features

Upon completion of the system, it is necessary to fully verify it, which was done. Scientists have assessed several fundamental factors: weight, latency, energy efficiency, blocking strength, accuracy of contact sensation, etc.

The weight of one locking module was 30 g. A block of 7 modules plus all the necessary electronics and mounts weigh 273 grams. The weight of all the elements worn on the user’s hand is only 11 grams. As a comparative example, scientists cite a portable controller HTC Vive, which weighs 203 g.

As for freedom of movement, the Wireality system allows you to contact virtual objects within a radius of 83 cm from the user’s shoulder.

Given the design of the system, each module must quickly stop the user’s hand in the right position, and this will have to be done repeatedly. Therefore, it was necessary to evaluate the maximum load force that the system can withstand. On average, this figure is 186 N.

The operation of the solenoids was also subjected to repeated testing, during which voltages in the range from 5 to 12 V were applied with a duration of 10 to 1000 ms. As we already know, it has been experimentally established that exactly 12 V and 40 ms are the perfect combination.

Solenoids and their response time also play an important role in shaping the energy efficiency of the entire system. Each interlock module consumes 2.19 W (183 mA at 12 V) when the solenoid is turned on. Given that the solenoid is turned on for only 40 ms to lock the ratchet, each lock event consumes only 0.024 mWh (0.088 J). As an example, scientists cite the Oculus Quest headset with a battery capacity of 14,000 mWh, which would be enough for half a million blocking events.

When a user wants to touch a virtual object, system delay is extremely important, i.e. the lower it is, the better. The total system delay was 29 ms: 9 ms – Leap Motion; 1 ms – serial communication; 4 ms – activation of the solenoid; 1 ms – ratchet and latch pairing; ~ 14 ms – response unit in the shoulder mount. In the future, it is planned to reduce these indicators.

The accuracy of determining the position of the user’s hand by the sensors and the corresponding response of the locking system was also evaluated. Of course, there is no perfect match between virtual and real coordinates, but the discrepancy does not exceed 1.8 cm at a speed of 30, 60 and 90 cm / s.

Practical experiences

Next, experiments were conducted in which 12 people took part (4 men and 8 women, average age 21 years). It is worth noting that 6 participants had never used VR systems before.

Each of the participants sat in the center of a 2×2 m room with an Oculus Rift headset. Three test options were implemented: without anything, a controller with vibration, and the Wireality system.

In each experiment, participants interacted with five objects: a wall; tilted by 45 ° flat surface; sphere sixth and irregular object (irregular shape, roughly speaking).

Image No. 6

The order of the objects and experience options was randomized for each participant. After the experiment, each participant filled out a short questionnaire, where he assessed his feelings about working in different conditions of the experiment on a scale from 1 (very bad) to 7 (excellent): realistic feeling of the object; comfort of use; freedom of movement.

Image No. 7

The graph above shows the results of practical experiments. Scientists note that they were fully prepared for the fact that the participants in the experiments will relatively low evaluate the Wireality system in terms of freedom and comfort. However, in this prototype, the main emphasis was on achieving maximum realism of sensations, and not on comfort, which can be improved in the future, paying attention to ergonomics.

Nevertheless, all participants in the experiments reacted extremely positively to the Wireality system. According to them, it allowed them to significantly evaluate objects in virtual reality.

Funny, some participants in the experiments believed that the blocking of the position of their hands would occur from the side of the objects (i.e., something would press on the hand, forming the outline of the object). Others hoped for a sense of texture of objects. Of course, this function does not exist in Wireality, but this is also a prototype.

Summing up the data from the experiments, scientists created several variants of virtual reality scenarios to demonstrate the capabilities of Wireality.

In total, four scenarios were implemented that reflect the possible use of Wireality: borders (objects of space limitations); large and heavy objects; interaction objects (buttons, levers, etc.); virtual characters.

For a more detailed acquaintance with the nuances of this work, I recommend that you look into report of scientists.


Virtual reality can give a person the opportunity to explore worlds that do not exist in reality, practice in the visual arts, adding three-dimensionality to his paintings, wander around museums located in distant countries, and much more. VR systems are not only games and entertainment, but also a new learning environment, and for someone (for example, people with disabilities), this is the only way to feel freedom of movement. In other words, VR can be implemented anywhere, if that makes sense, of course.

The only thing that virtual reality cannot do is, surprisingly, provide the reality of tactile interactions. The Wireality system solves this problem in a fairly creative way. Steel threads, ratchets, solenoids and so on are far from sci-fi futuristic details, however, in conjunction they made it possible to create a completely futuristic system. Of course, Wireality is still far from ideal, because you need to work on ergonomics, energy efficiency and latency of the system, which is what the authors of this invention plan to do in the future.

Thank you for your attention, stay curious and have a great weekend everyone, guys! 🙂

A bit of advertising 🙂

Thank you for staying with us. Do you like our articles? Want to see more interesting materials? Support us by placing an order or recommending to your friends, cloud VPS for developers from $ 4.99, A unique analogue of entry-level servers that was invented by us for you: The whole truth about VPS (KVM) E5-2697 v3 (6 Cores) 10GB DDR4 480GB SSD 1Gbps from $ 19 or how to divide the server correctly? (options are available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).

Dell R730xd 2 times cheaper at the Equinix Tier IV data center in Amsterdam? Only here 2 x Intel TetraDeca-Core Xeon 2x E5-2697v3 2.6GHz 14C 64GB DDR4 4x960GB SSD 1Gbps 100 TV from $ 199 in the Netherlands! Dell R420 – 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB – from $ 99! Read about How to Build Infrastructure Bldg. class using Dell R730xd E5-2650 v4 servers costing 9,000 euros for a penny?

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *