HoloUS: Augmented reality visualization of live ultrasound images using HoloLens for ultrasound-guided procedures

Nguyen, Trong; Plishker, William; Matisoff, Andrew; Sharma, Karun; Shekhar, Raj

doi:10.1007/s11548-021-02526-7

Cited by 35 publications

(20 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Low latencies are crucial for in situ visualization of US images especially when guiding interventions such as biopsies. In this work, the latency of displayed US images was sufficiently low for a real-time perception with a majority of frames being < 16 ms delayed, proving that our approach is faster than the 80 ms reported by Ngyuen et al [22]. García-Vázquez et al [11] stated a latency of 259 ms for US volumes, correspondingly a significantly higher amount of data was transmitted which makes a comparison difficult.…”

Section: Discussionmentioning

confidence: 53%

“…Some of them rely on outsidein approaches to track the US probe and display the image [8,9,20]. Existing works utilizing inside-out approaches are on the other hand performed using an AR marker [11,21,22]. To acquire and stream images from the US station to HoloLens, one can either use a frame grabber at the display output of the US station or, if available, a software development kit (SDK) providing access to the image data.…”

Section: Ultrasound Visualization With Hololensmentioning

confidence: 99%

See 1 more Smart Citation

UltrARsound: in situ visualization of live ultrasound images using HoloLens 2

Haxthausen

Moreta-Martínez

Pose-Díez-de-la-Lastra

et al. 2022

Int J CARS

View full text Add to dashboard Cite

Purpose Augmented Reality (AR) has the potential to simplify ultrasound (US) examinations which usually require a skilled and experienced sonographer to mentally align narrow 2D cross-sectional US images in the 3D anatomy of the patient. This work describes and evaluates a novel approach to track retroreflective spheres attached to the US probe using an inside-out technique with the AR glasses HoloLens 2. Finally, live US images are displayed in situ on the imaged anatomy. Methods The Unity application UltrARsound performs spatial tracking of the US probe and attached retroreflective markers using the depth camera integrated into the AR glasses—thus eliminating the need for an external tracking system. Additionally, a Kalman filter is implemented to improve the noisy measurements of the camera. US images are streamed wirelessly via the PLUS toolkit to HoloLens 2. The technical evaluation comprises static and dynamic tracking accuracy, frequency and latency of displayed images. Results Tracking is performed with a median accuracy of 1.98 mm/1.81$$^\circ $$ ∘ for the static setting when using the Kalman filter. In a dynamic scenario, the median error was 2.81 mm/1.70$$^\circ $$ ∘ . The tracking frequency is currently limited to 20 Hz. 83% of the displayed US images had a latency lower than 16 ms. Conclusions In this work, we showed that spatial tracking of retroreflective spheres with the depth camera of HoloLens 2 is feasible, achieving a promising accuracy for in situ visualization of live US images. For tracking, no additional hardware nor modifications to HoloLens 2 are required making it a cheap and easy-to-use approach. Moreover, a minimal latency of displayed images enables a real-time perception for the sonographer.

show abstract

Section: Discussionmentioning

confidence: 53%

Section: Ultrasound Visualization With Hololensmentioning

confidence: 99%

UltrARsound: in situ visualization of live ultrasound images using HoloLens 2

Haxthausen

Moreta-Martínez

Pose-Díez-de-la-Lastra

et al. 2022

Int J CARS

View full text Add to dashboard Cite

show abstract

“…Several groups reported the use of 2D information in the form of telestrations for telementoring in general surgical task training [ 58 ] or teleproctoring for assistance during neurovascular procedures [ 59 ]. Other uses of 2D data included the display of raw 2D ultrasound data for general intraprocedural guidance [ 83 , 92 ] and assistance during cervical pedicle screw placement [ 93 ].…”

Section: Resultsmentioning

confidence: 99%

“…There were several papers which reported the use of custom markers for surface-based tracking, though the performance did not significantly differ from ArUco or Vuforia tracking solutions [ 82 , 97 ]. In the surface-based tracking strategy, the square marker fiducials served to localize the patient and any tracked tools relative to the reference frame of the OST-HMD, allowing for the overlay of virtual content [ 33 , 46 , 88 , 91 , 92 ]. Other groups reported the use of retroreflective spheres and the infrared camera capabilities on the HoloLens 1, HoloLens 2, or custom devices for improved tracking accuracy and performance over monocular RGB led tracking [ 60 , 75 , 90 , 98 ].…”

Section: Resultsmentioning

confidence: 99%

“…Improved spatial awareness and mental mapping as compared with traditional guidance paradigms was reported by numerous articles across different surgical applications [ 33 , 46 , 55 , 75 , 97 ]. Reduced attention shift by using an OST-HMD for the display of guidance information was measured in fluoroscopic imaging for orthopedic surgery [ 81 ] and ultrasound imaging for ultrasound-guided procedures [ 92 ].…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

2022

View full text Add to dashboard Cite

We conducted a systematic review of recent literature to understand the current challenges in the use of optical see-through head-mounted displays (OST-HMDs) for augmented reality (AR) assisted surgery. Using Google Scholar, 57 relevant articles from 1 January 2021 through 18 March 2022 were identified. Selected articles were then categorized based on a taxonomy that described the required components of an effective AR-based navigation system: data, processing, overlay, view, and validation. Our findings indicated a focus on orthopedic (n=20) and maxillofacial surgeries (n=8). For preoperative input data, computed tomography (CT) (n=34), and surface rendered models (n=39) were most commonly used to represent image information. Virtual content was commonly directly superimposed with the target site (n=47); this was achieved by surface tracking of fiducials (n=30), external tracking (n=16), or manual placement (n=11). Microsoft HoloLens devices (n=24 in 2021, n=7 in 2022) were the most frequently used OST-HMDs; gestures and/or voice (n=32) served as the preferred interaction paradigm. Though promising system accuracy in the order of 2–5 mm has been demonstrated in phantom models, several human factors and technical challenges—perception, ease of use, context, interaction, and occlusion—remain to be addressed prior to widespread adoption of OST-HMD led surgical navigation.

show abstract