The Virtual Mirror: A New Interaction Paradigm for Augmented Reality Environments

Abstract

Medical augmented reality (AR) has been widely discussed within the medical imaging as well as computer aided surgery communities. Different systems for exemplary medical applications have been proposed. Some of them produced promising results. One major issue still hindering AR technology to be regularly used in medical applications is the interaction between physician and the superimposed 3-D virtual data. Classical interaction paradigms, for instance with keyboard and mouse, to interact with visualized medical 3-D imaging data are not adequate for an AR environment. This paper introduces the concept of a tangible/controllable Virtual Mirror for medical AR applications. This concept intuitively augments the direct view of the surgeon with all desired views on volumetric medical imaging data registered with the operation site without moving around the operating table or displacing the patient. We selected two medical procedures to demonstrate and evaluate the potentials of the Virtual Mirror for the surgical workflow. Results confirm the intuitiveness of this new paradigm and its perceptive advantages for AR-based computer aided interventions.

A novel 3D guidance system using augmented reality for percutaneous vertebroplasty

“…4,10 The results of the current study showed the error of the needle angle to be 0.96° in the axial plane when using a spine phantom. These experimental results were, however, measured 1.09 ± 0.78 ‡ * p < 0.01; EIA for both groups (without VIPAR vs with VIPAR; n = 10) and both operators were analyzed with Tukey test.…”

“…If AR technology was applied to navigation surgery such as spinal instrumentation placement or tumor resection, it could overcome the problem of current navigation systems that require the surgeon to deal with multiple areas of attention, the surgical site, and the display of the navigation system during the navigation task. 4 Furthermore, an AR system with a HMD can provide 3D information from any angle that the surgeon requires simply by moving the surgeon's head, and it can also provide a different augmented navigational view for each surgeon wearing an HMD. Augmented-reality guidance technology can also streamline the workflow of cumbersome spinal surgeries by allowing necessary medical data to be shared with each operator.…”

“…Bichlmeier and colleagues have proposed the use of AR for pedicle screw placement and reported superior accuracy of an AR navigation system with a head mount display (HMD) in experimental spine phantoms. 4 The authors of MRI-guided spinal biopsy studies using an AR system 8,10,25 and ARassisted thoracoscopy studies have also reported satisfactory accuracy using experimental models. 5,23 A novel 3D guidance system using augmented reality for percutaneous vertebroplasty Augmented reality (AR) is an imaging technology by which virtual objects are overlaid onto images of real objects captured in real time by a tracking camera.…”

Evaluation of the virtual mirror as a navigational aid for augmented reality driven minimally invasive procedures

“…The mirror was successfully tested with a phantom, a cadaver, and a patient. At that time, however, we have not evaluated its usefulness for any kind of application [2]. This first generation of the virtual mirror has been presented to 20 surgeons within the scope of an earlier study [20] that assessed the perception of different methods of in-situ visualization.…”

A Practical Setup for Projection-Based Augmented Maps

“…Finally, interaction models include both traditional computer-based controls and body mounted sensors or even more complex physical haptics. Recent approaches include wearable gestural interfaces [15], tangible interfaces [16] and virtual mirrors [17]. In this paper we develop a setup for projection-based augmented maps that is based on a projective-invariant marker design which is used for four different purposes: the detection of the physical controller moved by the user, the localization in the 3D space of the display surface and, finally, the calibration of both the camera and the projector.…”