An Interactive Mixed Reality Platform for Bedside Surgical Procedures

Azimi, Ehsan; Niu, Zhiyuan; Stiber, Maia; Greene, Nicholas D. E.; Liu, Ruby; Molina, Camilo A.; Huang, Judy; Huang, Chung‐Hao; Kazanzides, Peter

doi:10.1007/978-3-030-59716-0_7

Cited by 22 publications

(11 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An example is the use of outline rendering to show the misalignment of anatomical edges during keyhole surgery. 24 Azimi et al 3 provide a similar example of a neuro-surgical guidance system where the registration error is communicated to the surgeon via misalignment of external anatomy. Such a method could be relatively easily implemented in SciKit-SurgeryFRED, with gamification used to measure its effectiveness.…”

Section: Discussionmentioning

confidence: 99%

Are fiducial registration error and target registration error correlated? SciKit-SurgeryFRED for teaching and research

Thompson¹,

Dowrick

Ahmad

et al. 2021

Medical Imaging 2021: Image-Guided Procedures, Robotic Interventions, and Modeling

View full text Add to dashboard Cite

Understanding the relationship between fiducial registration error (FRE) and target registration error (TRE) is important for the correct use of interventional guidance systems. Whilst it is well established that TRE is statistically independent of FRE, system users still struggle against the intuitive assumption that a low FRE indicates a low TRE. We present the SciKit-Surgery Fiducial Registration Educational Demonstrator and describe its use. SciKit-SurgeryFRED was developed to enable remote teaching of key concepts in image registration. SciKit-SurgeryFRED also supports research into user interface design for image registration systems.SciKit-SurgeryFRED can be used to enable remote tutorials covering the statistics relevant to image guided interventions. Students are able to place fiducial markers on pre and intra-operative images and observe the effects of changes in marker geometry, marker count, and fiducial localisation error on TRE and FRE. SciKit-SurgeryFRED also calculates statistical measures for the expected values of TRE and FRE. Because many registrations can be performed quickly the students can then explore potential correlations between the different statistics.SciKit-SurgeryFRED also implements a registration based game, where participants are rewarded for complete treatment of a clinical target, whilst minimising the treatment margin. We used this game to perform a remote study on registration and simulated ablation, measuring how user performance changes depending on what error statistics are made available. The results support the assumption that knowing the exact value of target registration error leads to better treatment. Display of other statistics did not have a significant impact on the treatment performance.

show abstract

Section: Discussionmentioning

confidence: 99%

Are fiducial registration error and target registration error correlated? SciKit-SurgeryFRED for teaching and research

Thompson¹,

Dowrick

Ahmad

et al. 2021

Medical Imaging 2021: Image-Guided Procedures, Robotic Interventions, and Modeling

View full text Add to dashboard Cite

show abstract

“…Because of this fundamental difference between OST-based AR and video-based AR, the evaluation of the projected models' placements in the real world is different; since only users perceive the final output, traditional error measurements on 2D images are not enough. Generally, a user study is conducted to assess OSTbased AR systems (Azimi et al 2020).…”

Section: Introductionmentioning

confidence: 99%

Multimodal markers for technology-independent integration of augmented reality devices and surgical navigation systems

et al. 2022

View full text Add to dashboard Cite

Augmented reality (AR) permits the visualization of pre-operative data in the surgical field of view of the surgeon. This requires the alignment of the AR device’s coordinate system with the used navigation/tracking system. We propose a multimodal marker approach to align an AR device with a tracking system: in our implementation, an electromagnetic tracking system (EMTS). The solution makes use of a calibration method which determines the relationship between a 2D pattern detected by an RGB camera and an electromagnetic sensor of the EMTS. This allowed the projection of a 3D skull model on its physical counterpart. This projection was evaluated using a monocular camera and an optical see-through device (HoloLens 2) (https://www.microsoft.com/en-us/hololens/) achieving an accuracy of less than 2.5 mm in the image plane of the HoloLens 2 (HL2). Additionally, 10 volunteers participated in a user study consisting of an alignment task of a pointer with 25 projections viewed through the HL2. The participants achieved a mean error of 2.7 1.3 mm and 2.9 2.9$$^\circ $$ ∘ in positional and orientation error. This study showcases the feasibility of the approach, provides an evaluation of the alignment, and finally, discusses its advantages and limitations.

show abstract

“…Mixed Reality (MR)-based solutions can represent a low-cost, easily portable guidance system for EVD placement. Previous studies have shown the feasibility of adopting MR based solutions for ventriculostomy procedures, providing visualization of internal brain structures [5], or to register the manually segmented holographic content on the patient's anatomy using marker based methods [3,9,13]. Nonetheless, these approaches do not represent adequate solutions for emergency cases where both automatic segmentation and quick registration need to coexist in a fast multistep routine.…”

Section: Introductionmentioning

confidence: 99%

Mixed Reality and Deep Learning for External Ventricular Drainage Placement: A Fast and Automatic Workflow for Emergency Treatments

Palumbo

Saitta

Schiariti

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The treatment of hydrocephalus is based on anatomical landmarks to guide the insertion of an External Ventricular Drain (EVD). This procedure can benefit from the adoption of Mixed Reality (MR) technology. In this study, we assess the feasibility of a fully automatic MR and deep learning-based workflow to support emergency EVD placement, for which CT images are available and a fast and automatic workflow is needed. The proposed study provides a tool to automatically i) segment the skull, face skin, ventricles and Foramen of Monro from CT scans; ii) import the segmented model in the MR application; iii) register holograms on the patient's head via a marker-less approach. An ad-hoc evaluation approach including 3D-printed anatomical structures was developed to quantitatively assess the accuracy and usability of the registration workflow.

show abstract

An Interactive Mixed Reality Platform for Bedside Surgical Procedures

Cited by 22 publications

References 18 publications

Are fiducial registration error and target registration error correlated? SciKit-SurgeryFRED for teaching and research

Are fiducial registration error and target registration error correlated? SciKit-SurgeryFRED for teaching and research

Multimodal markers for technology-independent integration of augmented reality devices and surgical navigation systems

Mixed Reality and Deep Learning for External Ventricular Drainage Placement: A Fast and Automatic Workflow for Emergency Treatments

Contact Info

Product

Resources

About