Can extended reality technologies revolutionize medicine?
The talk about the amazing opportunities offered by augmented reality (AR) and virtual reality (VR) technologies has not yet died down, but another phrase has begun to be heard more and more often – “extended reality” (XR). The passion around it was, of course, ignited by the appearance of the Apple Vision Pro gadget. For some, these “glasses” have become an expensive toy, but such devices can also be used for serious useful things. For example, in medicine.
Layers of reality: understanding the terminology
For those who don’t know, it’s worth explaining all these AR, VR and XR. So:
Augmented reality (AR) – this is when a computer takes a picture of the real world and superimposes some useful (or not so useful) information on it. This can be done either using AR glasses or using a regular tablet or smartphone. If you remember the once sensational game Pokémon GO, this is it.
Virtual reality (VR) completely deceives the brain. It immerses a person in a completely fictitious, artificial world. Unlike AR, there is nothing left of reality at all.
Mixed reality (MR) allows real and digital objects to coexist and interact with each other. This is a more advanced thing compared to AR and VR, and it requires more serious computing power.
Finally, extended reality (XR) – this is, to put it simply, the coolest thing. It combines AR, VR and MR.
Experts predict a great future for XR. Here, for example, is what they wrote in the GlobalData report for July 2023:
XR has the potential to revolutionize learning, design, interaction, collaboration and entertainment. It will contribute to transformational changes in all industries and improve the quality of interaction between people.
Augmented reality for surgeons
Augmented reality technologies can lead to big changes in medicine. For example, there are already prerequisites for integrating XR into surgery. In early February 2024, as part of a collaboration between eXeX and neurosurgeon Robert Mason, XR was used during some spinal column reconstruction surgeries. The doctors noted that so far the technology is used in the operating room only “as a logistics and organizational tool.” But over time, the possibilities of its use will expand, and it can become an integral tool for surgeons.
Experts believe that during operations, XR features such as overlaying computer images and additional information on the field of view in real time, providing visual guidance, will be primarily useful. Outside of surgical interventions, extended reality can be used to analyze preoperative and postoperative data, allowing doctors to quickly access information about each patient.
Right now, several startups are introducing XR technologies into surgery. The leaders among them are Synaptive, XACT Robotics and CMR Surgical. The Australian company Omniscient creates digital maps of the brain, which can then be used by neurosurgeons, neurologists and psychotherapists.
Scale with HP above the patient's head
With Vision Pro, you can display any patient information in front of the doctor's eyes, from blood pressure graphs over time to the results of the latest chest x-ray or blood tests. The question is how necessary and advisable this is, and how much it will increase the efficiency of doctors’ work. In addition, before introducing such technologies into everyday clinical practice, it would be a good idea to thoroughly study their capabilities.
Sharp HealthCare, a nonprofit regional health care group in San Diego, recently began searching for answers to these questions. In early February, it purchased 30 Vision Pro kits and distributed them to healthcare workers performing a variety of tasks. To better explore how useful the new gadget will be in hospitals, Sharp is partnering with industry leader in electronic health records systems Epic.
Experts note that XR headsets certainly allow people to perceive information differently. But it is worth remembering that “different” and “better” are not synonymous. It is important to carefully compare how medical staff perform certain duties with and without the gadget to understand where augmented reality can be truly useful. The experiment involves not only doctors, but also nurses, information technology specialists and medical software developers.
Some applications for XR seem most obvious.
The Vision Pro headset has built-in internal video cameras aimed at the user's eyes. Due to the fact that they constantly track the gaze, a person can “select” an object or icon simply by looking at it. This can be very convenient for eye testing.
The use of XR looks promising when a specialist needs to solve several problems simultaneously. For example, a guard nurse in a hospital could quickly visualize the vital signs of the patients assigned to her, monitor through video cameras bedridden patients who can get up without permission and get injured.
Anesthesiologists are often forced to monitor a lot of indicators in real time. Instead of running around the wards or constantly looking at different monitors, the doctor could simply put on a headset and display all the necessary information through it.
In many computer games, there are scales above the characters' heads with the remaining HP, sometimes the level of energy and magic is displayed. Doctors could similarly see pulse rate, blood pressure, ECG results and other indicators above their patients' heads. At the same time, the classic phrase “examination of the patient” would sparkle with completely new colors!
Train on cats…virtual
3D virtual models are great for teaching medical students. Three-dimensional anatomical atlases on computers and smartphones are no longer a novelty. VR technologies significantly expand possibilities. With their help, doctors can show complex anatomical models. You can turn the virtual organ in different directions and see in detail how it works.
Using XR, you can go even further and simulate almost any medical procedure or operation. The trainee will be able to independently perform complex surgical interventions, and the cost of error will be zero.
Perhaps new technologies will help not only beginners. There are complex operations that have to be performed quite rarely, and the risk of postoperative complications and patient death is high. To master such interventions perfectly, a surgeon needs to work in a specialized clinic for quite a long time. And in the virtual space you could train every day.
Medical interventions using VR can be demonstrated to the patient himself. Then he will have a better idea of what he has to endure, will begin to cooperate more consciously and conscientiously with the doctor, and with the right approach, this will significantly reduce the level of stress.
Will local therapists start seeing patients wearing XR glasses?
Well, the main question: Will we soon see therapists and other doctors who will see patients using an XR headset? Perhaps this would be convenient for the doctors themselves. But experts believe that such use of technology is inappropriate.
Still, during an appointment, human interaction between the doctor and the patient plays a big role. It is difficult to talk about trusting relationships if a masked specialist who resembles a robot is sitting in front of you. This problem has already been discussed during pandemic times, when many patients had to communicate with a respirator and safety glasses instead of the usual human face.
Whether it is a matter of quickly obtaining the necessary information in the operating room, intensive care unit, or before the patient enters the office. Here the benefits of such gadgets are obvious – all that remains is to understand how to use them most rationally. There is no doubt that this direction will develop, and the market for XR technologies in medicine will only grow.
