Evaluation of Different Visualization Techniques for Perception-Based Alignment in Medical AR

Fischer, Molly; Leuze, Christoph; Perkins, Stephanie L.; Rosenberg, Jarrett; Daniel, Bruce L.; Martin-Gomez, Alejandro

doi:10.1109/ismar-adjunct51615.2020.00027

Cited by 15 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After preoperative imaging, Digital Imaging and Communications in Medicine (DICOM) formatted data are typically provided which can be read in an open source library, such as 3D Slicer ( , accessed on 23 March 2022) [ 46 , 65 , 74 , 87 ], or proprietary software, such as Mimics ( , accessed on 23 March 2022) [ 47 , 61 , 88 ] or Brainlab ( , accessed on 23 March 2022) [ 67 , 84 ]. Surface rendered models provide enormous utility due to their ease of creation, lightweight model representation, and additional modifiable visualization properties, such as color, transparency, wireframe representations, and heatmaps [ 89 ]. For performance limited devices, such as current commercially available OST-HMDs, the number of vertices used to represent a 3D virtual model needs to be limited to ensure a performant application; many groups have mentioned the simplification of their model using open source tools, such as Blender ( , accessed on 23 March 2022) and MeshLab ( , accessed on 23 March 2022) [ 61 ].…”

Section: Resultsmentioning

confidence: 99%

“…[47,61,88] or Brainlab (https://www.brainlab.com/digital-o-r/surgical-planning/, accessed on 23 March 2022) [67,84]. Surface rendered models provide enormous utility due to their ease of creation, lightweight model representation, and additional modifiable visualization properties, such as color, transparency, wireframe representations, and heatmaps [89]. For performance limited devices, such as current commercially available OST-HMDs, the number of vertices used to represent a 3D virtual model needs to be limited to ensure a performant application; many groups have mentioned the simplification of their model using open source tools, such as Blender (https://www.blender.org/, accessed on 23 March 2022) and MeshLab (https://www.meshlab.net/, accessed on 23 March 2022) [61].…”

Section: Processingmentioning

confidence: 99%

See 1 more Smart Citation

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

2022

View full text Add to dashboard Cite

We conducted a systematic review of recent literature to understand the current challenges in the use of optical see-through head-mounted displays (OST-HMDs) for augmented reality (AR) assisted surgery. Using Google Scholar, 57 relevant articles from 1 January 2021 through 18 March 2022 were identified. Selected articles were then categorized based on a taxonomy that described the required components of an effective AR-based navigation system: data, processing, overlay, view, and validation. Our findings indicated a focus on orthopedic (n=20) and maxillofacial surgeries (n=8). For preoperative input data, computed tomography (CT) (n=34), and surface rendered models (n=39) were most commonly used to represent image information. Virtual content was commonly directly superimposed with the target site (n=47); this was achieved by surface tracking of fiducials (n=30), external tracking (n=16), or manual placement (n=11). Microsoft HoloLens devices (n=24 in 2021, n=7 in 2022) were the most frequently used OST-HMDs; gestures and/or voice (n=32) served as the preferred interaction paradigm. Though promising system accuracy in the order of 2–5 mm has been demonstrated in phantom models, several human factors and technical challenges—perception, ease of use, context, interaction, and occlusion—remain to be addressed prior to widespread adoption of OST-HMD led surgical navigation.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Processingmentioning

confidence: 99%

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

2022

View full text Add to dashboard Cite

show abstract

“…Using a marker-based alignment approach, the perceptual limitations of OST-HMDs, due to contribution of the vergence-accommodation conflict, have been investigated in guidance during manual tasks, with superimposed virtual content, using the HoloLens 1 [18] and Magic Leap One [19]. In a non-marker-based alignment strategy, other groups have focused on investigating how different visualization techniques or a multi-view AR experience can influence a users' ability to perform perception-based manual alignment of virtual-to-real content for breast reconstruction surgery [20] or robot-assisted minimally invasive surgery [21].…”

Section: Related Workmentioning

confidence: 99%

Head-Mounted Display-Based Augmented Reality for Image-Guided Media Delivery to the Heart: A Preliminary Investigation of Perceptual Accuracy

Doughty

Ghugre

2022

J. Imaging

View full text Add to dashboard Cite

By aligning virtual augmentations with real objects, optical see-through head-mounted display (OST-HMD)-based augmented reality (AR) can enhance user-task performance. Our goal was to compare the perceptual accuracy of several visualization paradigms involving an adjacent monitor, or the Microsoft HoloLens 2 OST-HMD, in a targeted task, as well as to assess the feasibility of displaying imaging-derived virtual models aligned with the injured porcine heart. With 10 participants, we performed a user study to quantify and compare the accuracy, speed, and subjective workload of each paradigm in the completion of a point-and-trace task that simulated surgical targeting. To demonstrate the clinical potential of our system, we assessed its use for the visualization of magnetic resonance imaging (MRI)-based anatomical models, aligned with the surgically exposed heart in a motion-arrested open-chest porcine model. Using the HoloLens 2 with alignment of the ground truth target and our display calibration method, users were able to achieve submillimeter accuracy (0.98 mm) and required 1.42 min for calibration in the point-and-trace task. In the porcine study, we observed good spatial agreement between the MRI-models and target surgical site. The use of an OST-HMD led to improved perceptual accuracy and task-completion times in a simulated targeting task.

show abstract

“…Although the AR system can effectively address some problems in present medical data visualisation, its own issues in image perception processing following the virtual‐real fusion, particularly in scene depth perception, have yet to be resolved. This is also a critical issue in determining whether AR systems can be widely employed in the medical industry and offer high accuracy and safety for surgical procedures, and improving the user's image perception problem is one of the main AR system research objectives at the moment 29,36 . The depth information provided by AR images enables doctors to determine the spatial relationship between real and virtual objects, allowing them to better immerse themselves in the system's augmented environment.…”

Section: Introductionmentioning

confidence: 99%

“…This is also a critical issue in determining whether AR systems can be widely employed in the medical industry and offer high accuracy and safety for surgical procedures, and improving the user's image perception problem is one of the main AR system research objectives at the moment. 29,36 The depth information provided by AR images enables doctors to determine the spatial relationship between real and virtual objects, allowing them to better immerse themselves in the system's augmented environment. If doctors perceive image depth incorrectly, they may be unable to correctly match virtual information with the real world, which can have a significant impact on the correct recognition of complex structures.…”

Section: Introductionmentioning

confidence: 99%

User's image perception improved strategy and application of augmented reality systems in smart medical care: A review

Jiang

Zhang

Sun

et al. 2023

Robotics Computer Surgery

View full text Add to dashboard Cite

Background: Augmented reality (AR) is a new human-computer interaction technology that combines virtual reality, computer vision, and computer networks. With the rapid advancement of the medical field towards intelligence and data visualisation, AR systems are becoming increasingly popular in the medical field because they can provide doctors with clear enough medical images and accurate image navigation in practical applications. However, it has been discovered that different display types of AR systems have different effects on doctors' perception of the image after virtual-real fusion during the actual medical application. If doctors cannot correctly perceive the image, they may be unable to correctly match the virtual information with the real world, which will have a significant impact on their ability to recognise complex structures. Methods: This paper uses Citespace, a literature analysis tool, to visualise and analyse the research hotspots when AR systems are used in the medical field. Results: A visual analysis of the 1163 articles retrieved from the Web of Science Core Collection database reveals that display technology and visualisation technology are the key research directions of AR systems at the moment. Conclusion: This paper categorises AR systems based on their display principles, reviews current image perception optimisation schemes for various types of systems, and analyses and compares different display types of AR systems based on their practical applications in the field of smart medical care so that doctors can select the appropriate display types based on different application scenarios.

show abstract

Evaluation of Different Visualization Techniques for Perception-Based Alignment in Medical AR

Cited by 15 publications

References 11 publications

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

Head-Mounted Display-Based Augmented Reality for Image-Guided Media Delivery to the Heart: A Preliminary Investigation of Perceptual Accuracy

User's image perception improved strategy and application of augmented reality systems in smart medical care: A review

Contact Info

Product

Resources

About