Augmented reality-guided localization of a chest wall tumor in a pediatric patient

Spijkerboer, Koen; Fitski, Matthijs; Siepel, Françoise J.; Ven, Cornelis P. van de; Steeg, Alida F. W. van der

doi:10.1016/j.ejca.2022.04.023

Cited by 4 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Portable devices like smart mobile phones and tablet computers are easy to be carried with users and could be better at protecting privacy while fixed devices like public screens could realize teamwork based on the same vision. In medicine, a good application is that the tumor model could be projected to the screen of the thoracoscope according to the CT or MRI scan before the surgery, which is beneficial for delineating the area of excision for the thoracic surgical team and reducing the loss of normal tissue (18). However, the disadvantages are also obvious: lack of immersed feeling, lower rendering quality, and registration degree.…”

Section: The Window-based Displaymentioning

confidence: 99%

The application and prospection of augmented reality in hepato-pancreato-biliary surgery

Dai

Qiu

et al. 2023

BST

View full text Add to dashboard Cite

Augmented Reality (AR) is one of the main forms of Extended Reality (XR) application in surgery. hepato-pancreato-biliary (HPB) surgeons could benefit from AR as an efficient tool for making surgical plans, providing intraoperative navigation, and enhancing surgical skills. The introduction of AR to HPB surgery is less than 30 years but brings profound influence. From the early days of projecting liver models on patients' surfaces for locating a better puncture point to today's assisting surgeons to perform live donor liver transplantation, a series of successful clinical practices have proved that AR can play a constructive role in HPB surgery and has great potential. Thus far, AR has been shown to increase efficiency and safety in surgical resection, and, at the same time, can improve oncological outcomes and reduce surgical risk. Although AR has presented admitted advantages in surgery, AR's application is still immature as an emerging technique and needs more exploration. In this paper, we reviewed the principles of AR and its developing history in HPB surgery, describing its significant practical applications over the past 30 years. Reviewing the past attempts of AR in HPB surgery could make HPB surgeons a better understanding of future surgery and the digital trends in medicine. The routine uses of AR in HPB surgery, as an indication of the operating room entering the new era, is coming soon.

show abstract

Section: The Window-based Displaymentioning

confidence: 99%

The application and prospection of augmented reality in hepato-pancreato-biliary surgery

Dai

Qiu

et al. 2023

BST

View full text Add to dashboard Cite

show abstract

“…After reconstructing MRI data into 3D models [ 11 ], the 3D holographic models can be projected onto the surgical region within the surgeon’s field of view using an AR-based tracking camera integrated head-mounted display (HMD) [ 12 ]. The use of these 3D visualizations is expected to improve the understanding of precise tumor locations and aid in the assessment of vital anatomical structures, including arteries, veins, and nerves [ 13 , 14 ].…”

Section: Introductionmentioning

confidence: 99%

Clinical evaluation of augmented reality-based 3D navigation system for brachial plexus tumor surgery

Zhao,

Zheng

et al. 2024

World J Surg Onc

View full text Add to dashboard Cite

Background Augmented reality (AR), a form of 3D imaging technology, has been preliminarily applied in tumor surgery of the head and spine, both are rigid bodies. However, there is a lack of research evaluating the clinical value of AR in tumor surgery of the brachial plexus, a non-rigid body, where the anatomical position varies with patient posture. Methods Prior to surgery in 8 patients diagnosed with brachial plexus tumors, conventional MRI scans were performed to obtain conventional 2D MRI images. The MRI data were then differentiated automatically and converted into AR-based 3D models. After point-to-point relocation and registration, the 3D models were projected onto the patient’s body using a head-mounted display for navigation. To evaluate the clinical value of AR-based 3D models compared to the conventional 2D MRI images, 2 senior hand surgeons completed questionnaires on the evaluation of anatomical structures (tumor, arteries, veins, nerves, bones, and muscles), ranging from 1 (strongly disagree) to 5 (strongly agree). Results Surgeons rated AR-based 3D models as superior to conventional MRI images for all anatomical structures, including tumors. Furthermore, AR-based 3D models were preferred for preoperative planning and intraoperative navigation, demonstrating their added value. The mean positional error between the 3D models and intraoperative findings was approximately 1 cm. Conclusions This study evaluated, for the first time, the clinical value of an AR-based 3D navigation system in preoperative planning and intraoperative navigation for brachial plexus tumor surgery. By providing more direct spatial visualization, compared with conventional 2D MRI images, this 3D navigation system significantly improved the clinical accuracy and safety of tumor surgery in non-rigid bodies.

show abstract

“…Augmented reality (AR) refers to one of the techniques in computer vision that overlays virtual images on those of the real world. Several studies have attempted to provide images of 3D objects through AR, validating their performance in the fields of medical assistance and diagnosis [22][23][24][25]. Within the pre-installed markers and tracking conditions, AR provides fast and precise guidance in environments where targeted objects, such as swabs, are hidden behind other objects.…”

Section: Introductionmentioning

confidence: 99%

Deep Neural Network-Based Visual Feedback System for Nasopharyngeal Swab Sampling

Jung,

Moon,

Kim

et al. 2023

Sensors

View full text Add to dashboard Cite

During the 2019 coronavirus disease pandemic, robotic-based systems for swab sampling were developed to reduce burdens on healthcare workers and their risk of infection. Teleoperated sampling systems are especially appreciated as they fundamentally prevent contact with suspected COVID-19 patients. However, the limited field of view of the installed cameras prevents the operator from recognizing the position and deformation of the swab inserted into the nasal cavity, which highly decreases the operating performance. To overcome this limitation, this study proposes a visual feedback system that monitors and reconstructs the shape of an NP swab using augmented reality (AR). The sampling device contained three load cells and measured the interaction force applied to the swab, while the shape information was captured using a motion-tracking program. These datasets were used to train a one-dimensional convolution neural network (1DCNN) model, which estimated the coordinates of three feature points of the swab in 2D X–Y plane. Based on these points, the virtual shape of the swab, reflecting the curvature of the actual one, was reconstructed and overlaid on the visual display. The accuracy of the 1DCNN model was evaluated on a 2D plane under ten different bending conditions. The results demonstrate that the x-values of the predicted points show errors of under 0.590 mm from P0, while those of P1 and P2 show a biased error of about −1.5 mm with constant standard deviations. For the y-values, the error of all feature points under positive bending is uniformly estimated with under 1 mm of difference, when the error under negative bending increases depending on the amount of deformation. Finally, experiments using a collaborative robot validate its ability to visualize the actual swab’s position and deformation on the camera image of 2D and 3D phantoms.

show abstract

Augmented reality-guided localization of a chest wall tumor in a pediatric patient

Cited by 4 publications

References 9 publications

The application and prospection of augmented reality in hepato-pancreato-biliary surgery

The application and prospection of augmented reality in hepato-pancreato-biliary surgery

Clinical evaluation of augmented reality-based 3D navigation system for brachial plexus tumor surgery

Deep Neural Network-Based Visual Feedback System for Nasopharyngeal Swab Sampling

Contact Info

Product

Resources

About