Utility of optical see-through head mounted displays in augmented reality-assisted surgery: A systematic review

Birlo, Manuel; Edwards, Philip; Clarkson, Matthew J.; Stoyanov, Danail

doi:10.1016/j.media.2022.102361

Cited by 65 publications

(49 citation statements)

References 126 publications

(546 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The final aspect of system validation which was documented in the articles was related to human factors and workflow benefits and limitations ( Table 6 ). Human factors considerations have been consistently reported as one of the key challenges limiting the uptake of OST-HMD-based surgical navigation into the clinic [ 6 ]. In our assessment of OST-HMD-based surgical navigation systems, we categorized each reported human factors consideration as being: (1) addressed; (2) partially addressed; or (3) remaining to be addressed (persistent).…”

Section: Resultsmentioning

confidence: 99%

“…Mental mapping is closely associated with spatial awareness and refers to the requirement for a surgeon to project (or mentally map) 2D image data onto the 3D scene in order to leverage this information for surgical guidance. A reliance on mental projection of data for surgical guidance can introduce errors to the surgery and additional mental workload on the surgeon [ 6 , 33 ]. Attention shift describes the requirement for a surgeon to frequently gaze between the provided navigation information on an external display and the patient; with an OST-HMD, virtual guidance information can be brought directly into the FOV of the wearer to remove this requirement.…”

Section: Resultsmentioning

confidence: 99%

“…Although there have been significant efforts to improve the design of OST-HMD led surgical navigation platforms, applications have remained constrained to research lab environments and there has been little clinical uptake [ 5 , 6 ]. As evidenced by the broad survey of recent literature, the poor clinical uptake of OST-HMDs for surgical guidance can be partially attributed to a lack of HMD performance [ 7 ], rendering resolution [ 8 ], and application or task-specific HMD design [ 21 ]; however, perceptual challenges [ 9 , 10 , 33 ], surgical context and interaction limitations [ 11 , 12 ], and registration and occlusion challenges [ 13 , 14 ] remain the key hurdles in the widespread clinical adoption of these technologies.…”

Section: Resultsmentioning

confidence: 99%

“…In Figure 6 , we include a chart depicting the article frequency categorized by the publication year. For the years 2014 to 2020, we include the data provided in a recent review of OST-HMD indications in surgery by Birlo et al [ 6 ]. In our review, we used similar search terms and criteria as Birlo et al to provide comparable data.…”

Section: Methodsmentioning

confidence: 99%

“…An uptick in the frequency of publications involving the use of OST-HMDs for surgical applications is visible and coincides with the introduction of commercially available HMDs: the first edition Google Glass OST-HMD in 2014 and the Microsoft HoloLens 1 OST-HMD in 2016. We have included the count provided by Birlo et al for the years 2014 to 2020 [6] in blue with our contributions in green. As the review was conducted in the first quarter of 2022, an estimate of the full-year article publication count was arrived at by multiplying the count by four.…”

Section: Literature Search Strategymentioning

confidence: 99%

See 4 more Smart Citations

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

2022

View full text Add to dashboard Cite

We conducted a systematic review of recent literature to understand the current challenges in the use of optical see-through head-mounted displays (OST-HMDs) for augmented reality (AR) assisted surgery. Using Google Scholar, 57 relevant articles from 1 January 2021 through 18 March 2022 were identified. Selected articles were then categorized based on a taxonomy that described the required components of an effective AR-based navigation system: data, processing, overlay, view, and validation. Our findings indicated a focus on orthopedic (n=20) and maxillofacial surgeries (n=8). For preoperative input data, computed tomography (CT) (n=34), and surface rendered models (n=39) were most commonly used to represent image information. Virtual content was commonly directly superimposed with the target site (n=47); this was achieved by surface tracking of fiducials (n=30), external tracking (n=16), or manual placement (n=11). Microsoft HoloLens devices (n=24 in 2021, n=7 in 2022) were the most frequently used OST-HMDs; gestures and/or voice (n=32) served as the preferred interaction paradigm. Though promising system accuracy in the order of 2–5 mm has been demonstrated in phantom models, several human factors and technical challenges—perception, ease of use, context, interaction, and occlusion—remain to be addressed prior to widespread adoption of OST-HMD led surgical navigation.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Literature Search Strategymentioning

confidence: 99%

See 3 more Smart Citations

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

2022

View full text Add to dashboard Cite

show abstract

Successful laparoscopic retroperitoneal tumor resection using mixed reality and guiding marker techniques

Tohi,

Okazoe,

Mitamura

et al. 2024

IJU Case Reports

View full text Add to dashboard Cite

IntroductionSmall tumors may be difficult to identify visually and require preoperative effort to locate. Recent advancements in mixed reality technology have improved surgical accuracy in various departments. Here, we present the application of mixed reality‐assisted surgery and a guiding marker in the case of small retroperitoneal metastasis of uterine cancer.Case presentationA 67‐year‐old female with a history of uterine cancer had a retroperitoneal metastasis in the lateroconal fascia near the right diaphragm, measuring 2 cm and infiltrating the peritoneum. We performed precise surgical planning using the preoperative mixed reality software “Holoeyes” on a head‐mounted display called HoloLens2. Novel techniques, including ultrasonography‐guided placement of a guiding marker and strategic port‐site placement facilitated by HoloLens2, ensured accurate tumor identification and laparoscopic resection with minimal blood loss and no intraoperative complications.ConclusionThe use of mixed reality‐assisted surgery and a guiding marker effectively enhanced the precision of retroperitoneal tumor resection.

show abstract

R‐MFE‐TCN: A correlation prediction model between body surface and tumor during respiratory movement

Wang,

Yang,

Liu

et al. 2024

Medical Physics

View full text Add to dashboard Cite

Background2D CT image‐guided radiofrequency ablation (RFA) is an exciting minimally invasive treatment that can destroy liver tumors without removing them. However, CT images can only provide limited static information, and the tumor will move with the patient's respiratory movement. Therefore, how to accurately locate tumors under free conditions is an urgent problem to be solved at present.PurposeThe purpose of this study is to propose a respiratory correlation prediction model for mixed reality surgical assistance system, Riemannian and Multivariate Feature Enhanced Temporal Convolutional Network (R‐MFE‐TCN), and to achieve accurate respiratory correlation prediction.MethodsThe model adopts a respiration‐oriented Riemannian information enhancement strategy to expand the diversity of the dataset. A new Multivariate Feature Enhancement module (MFE) is proposed to retain respiratory data information, so that the network can fully explore the correlation of internal and external data information, the dual‐channel is used to retain multivariate respiratory feature, and the Multi‐headed Self‐attention obtains respiratory peak‐to‐valley value periodic information. This information significantly improves the prediction performance of the network. At the same time, the PSO algorithm is used for hyperparameter optimization. In the experiment, a total of seven patients' internal and external respiratory motion trajectories were obtained from the dataset, and the first six patients were selected as the training set. The respiratory signal collection frequency was 21 Hz.ResultsA large number of experiments on the dataset prove the good performance of this method, which improves the prediction accuracy while also having strong robustness. This method can reduce the delay deviation under long window prediction and achieve good performance. In the case of 400 ms, the average RMSE and MAE are 0.0453 and 0.0361 mm, respectively, which is better than other research methods.ConclusionThe R‐MFE‐TCN can be extended to respiratory correlation prediction in different clinical situations, meeting the accuracy requirements for respiratory delay prediction in surgical assistance.

show abstract

Utility of optical see-through head mounted displays in augmented reality-assisted surgery: A systematic review

Cited by 65 publications

References 126 publications

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

Successful laparoscopic retroperitoneal tumor resection using mixed reality and guiding marker techniques

R‐MFE‐TCN: A correlation prediction model between body surface and tumor during respiratory movement

Contact Info

Product

Resources

About