Concepts for augmented reality visualisation to support needle guidance inside the MRI

Abstract

During MRI-guided interventions, navigation support is often separated from the operating field on displays, which impedes the interpretation of positions and orientations of instruments inside the patient's body as well as hand–eye coordination. To overcome these issues projector-based augmented reality can be used to support needle guidance inside the MRI bore directly in the operating field. The authors present two visualisation concepts for needle navigation aids which were compared in an accuracy and usability study with eight participants, four of whom were experienced radiologists. The results show that both concepts are equally accurate ( and ), useful and easy to use, with clear visual feedback about the state and success of the needle puncture. For easier clinical applicability, a dynamic projection on moving surfaces and organ movement tracking are needed. For now, tests with patients with respiratory arrest are feasible.

HoloInjection: augmented reality support for CT‐guided spinal needle injections

“…Video see-through AR experiences, created with monitors [12] and head-mounted displays [9], were developed to superimpose camera views on the injection site with needle guidance aids or radiological images. Projective AR approaches were used to project-specific navigation instructions on how to position and insert instruments [8,13]. Optical see-through AR solutions enabled the superposition of guidance information through semi-transparent displays [10] or AR glasses like the HoloLens [14].…”

“…Heinrich et al [11] analysed prevalent navigation visualisation methods and conclude that they can be vaguely clustered into seethrough vision, access path and explicit navigation aids concepts. The latter was mainly adapted for projective AR and required regularly updated instrument tracking information to calculate current navigation instructions [8,11,13]. Besides tracking hardware, such systems also require stably mounted and sophisticatedly calibrated projectors, which may not be applicable for clinical routine.…”

“…See‐through vision concepts enable the view through the patient's skin by visualising correctly registered anatomical structures or radiological images together with information on current instrument positions. Such concepts were realised for projective AR [13], video see‐through AR [9] and optical see‐through AR [10]. However, these systems consist of time‐consuming to set up hardware devices.…”

Augmented reality technology in image-guided therapy: State-of-the-art review

“…2,3 In order to further improve outcomes from imageguided therapy, clinicians often combine other navigation methods to locate target lesions faster and more accurately so that they can apply surgical instruments accurately and avoid delicate structures. 4 The operators of image-guided therapy have to shift their focus between the surgical field and the display screen showing the real-time information. The need for mental registration of different image modalities is challenging since it has a high-level requirement for the operator to look at the position of the target in the image and determine the corresponding location in the patient.…”

“…Unfortunately, the difficulty and subjectivity of this process increase the risks to patients. 4,5 Both virtual reality (VR) and augmented reality (AR) technology have been used in attempts to overcome this problem.…”