Medical Augmented-Reality Visualizer for Surgical Training and Education in Medicine

Abstract

This paper presents a projection-based augmented-reality system (MARVIS) that supports the visualization of internal structures on the surface of a liver phantom. MARVIS is endowed with three key features: tracking of spatial relationship between the phantom and the operator’s head in real time, monoscopic projection of internal liver structures onto the phantom surface for 3D perception without additional head-mounted devices, and phantom internal electronic circuit to assess the accuracy of a syringe guidance system. An initial validation was carried out by 25 medical students (12 males and 13 females; mean age, 23.12 years; SD, 1.27 years) and 3 male surgeons (mean age, 43.66 years; SD, 7.57 years). The validation results show that the ratio of failed syringe insertions was reduced from 50% to 30% by adopting the MARVIS projection. The proposed system suitably enhances a surgeon’s spatial perception of a phantom internal structure.

Current Perspectives on Augmented Reality in Medical Education: Applications, Affordances and Limitations

“…A common use of AR in these studies is in conjunction with physical phantoms (such as silicon models of human organs) 30 , 37 and other simulation equipment that enables medical students to practice activities such as inserting needles into patients. 31 These types of studies seem overall to show more success than those where AR is just a visualization that is not linked to another physical artefact. This emphasizes the importance of the reality-virtuality bridge that augmented reality can provide.…”

“… 9. Gierwiało et al 31 2019 Surgery (liver syringe insertions) Participants simulated a liver biopsy and thermoablation in two conditions. Accuracy of the needle position was assessed 25 medical students and 3 surgeons The ratio of failed syringe insertions was reduced from 50% to 30% by using the AR tool.…”

Applicability of Augmented Reality in an Organ Transplantation

“…The authors of this manuscript have developed another experimental AR system, MARVIS (Medical AR VISualizer) [ 25 ], which does not fit into any of the aforementioned categories. We used spatial recognition of the phantom surface and the operator’s head position in real time.…”

“…The monoscopic projector hanging over an operating field displays the CT-acquired AR image of the target lesion hidden inside the phantom on its uneven surface, as seen from the operator’s point of view. Head movement changes the view angle and shape of the displayed image, enabling an operator to place the needle precisely in the hidden target ( Figure 1 ) [ 25 ].…”

Projected Augmented Reality to Drive Osteotomy Surgery: Implementation and Comparison With Video See-Through Technology

“…In recent years, a different technology for AR has started to make its way into the current medical literature: it is based on video projectors that, when properly calibrated with a tracking system, can provide AR information directly on the structure of interest, overcoming the perceptual limitations introduced above owing to focal rivalry and vergenceaccommodation conflict. This is the case, for example, of Mewes at al [13], who used an ultra-long-throw projector to guide the radiologist during interventional magnetic resonance imaging; or Gierwialo at al [14], who applied projected AR in surgical training and education, allowing medical students to visualize the internal structures of a liver mannequin. Both of these works rely on the use of heavy and cumbersome projectors, and for their successful integration into the operating room, they require a review and reorganization of the equipment layout.…”