A Bronchoscope Localization Method Using an Augmented Reality Co-Display of Real Bronchoscopy Images with a Virtual 3D Bronchial Tree Model

Chien, Jong-Chih; Lee, Jiann-Der; Su, Ellen; Li, Shih‐Hong

doi:10.3390/s20236997

Cited by 8 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Video bronchoscopy has been an important diagnostic or interventional tool for anesthesiology as well as pulmonary and critical care medicine. Although video bronchoscopy is a safe method, accurate navigation through airways requires thorough training 28 , 29 . The Accreditation Council for Graduate Medical Education, American College of Chest Physicians, American Thoracic Society, and European Respiratory Society require a certain number of procedures for demonstrating competence in interpretation of examination results 30 , 31 .…”

Section: Discussionmentioning

confidence: 99%

“…Apart from anesthesiologic use, interventional video bronchoscopy, including biopsy, anatomical navigation is more important for targeting appropriate tissue location. Thus, several technologies, including augmented reality and 3D printing, have been used to support video bronchoscopy training 10 , 28 , 34 , 35 . However, those studies required prebronchoscopic computed tomography scan to construct 3D segmented volumes and additional display devices.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Deep learning for anatomical interpretation of video bronchoscopy images

Yoo

Kang

Cho

et al. 2021

Sci Rep

View full text Add to dashboard Cite

Anesthesiologists commonly use video bronchoscopy to facilitate intubation or confirm the location of the endotracheal tube; however, depth and orientation in the bronchial tree can often be confused because anesthesiologists cannot trace the airway from the oropharynx when it is performed using an endotracheal tube. Moreover, the decubitus position is often used in certain surgeries. Although it occurs rarely, the misinterpretation of tube location can cause accidental extubation or endobronchial intubation, which can lead to hyperinflation. Thus, video bronchoscopy with a decision supporting system using artificial intelligence would be useful in the anesthesiologic process. In this study, we aimed to develop an artificial intelligence model robust to rotation and covering using video bronchoscopy images. We collected video bronchoscopic images from an institutional database. Collected images were automatically labeled by an optical character recognition engine as the carina and left/right main bronchus. Except 180 images for the evaluation dataset, 80% were randomly allocated to the training dataset. The remaining images were assigned to the validation and test datasets in a 7:3 ratio. Random image rotation and circular cropping were applied. Ten kinds of pretrained models with < 25 million parameters were trained on the training and validation datasets. The model showing the best prediction accuracy for the test dataset was selected as the final model. Six human experts reviewed the evaluation dataset for the inference of anatomical locations to compare its performance with that of the final model. In the experiments, 8688 images were prepared and assigned to the evaluation (180), training (6806), validation (1191), and test (511) datasets. The EfficientNetB1 model showed the highest accuracy (0.86) and was selected as the final model. For the evaluation dataset, the final model showed better performance (accuracy, 0.84) than almost all human experts (0.38, 0.44, 0.51, 0.68, and 0.63), and only the most-experienced pulmonologist showed performance comparable (0.82) with that of the final model. The performance of human experts was generally proportional to their experiences. The performance difference between anesthesiologists and pulmonologists was marked in discrimination of the right main bronchus. Using bronchoscopic images, our model could distinguish anatomical locations among the carina and both main bronchi under random rotation and covering. The performance was comparable with that of the most-experienced human expert. This model can be a basis for designing a clinical decision support system with video bronchoscopy.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Deep learning for anatomical interpretation of video bronchoscopy images

Yoo

Kang

Cho

et al. 2021

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Traditionally, the problem of image-based tracking in VNB has been solved through geometric and handcrafted methods. Feature generation (Chien et al, 2020;Byrnes and Higgins, 2014;? ;Luó et al, 2012a) and similarity measures (Luo and Mori, 2014;Shen et al, 2015;Luo et al, 2011;Khare and Higgins, 2010) were commonly used for such purpose, accounting, however, with tracking errors and large execution times.…”

Section: Methodsmentioning

confidence: 99%

“…Therefore, during navigation, VNB models require the registration of the virtual lung model with the frames from the videobronchoscopy to provide an effective guidance during the biopsy. The registration can be achieved by, either tracking the position and orientation of the bronchoscopy camera (Chien et al, 2020;Byrnes and Higgins, 2014;Luó et al, 2011Luó et al, , 2012a, or by callibrating its deviation from the pose (position and orientation) simulated in the virtual lung model.…”

Section: Introductionmentioning

confidence: 99%

BronchoPose: an analysis of data and model configuration for vision-based bronchoscopy pose estimation

Borrego-Carazo¹,

Sánchez²,

Castells-Rufas³

et al. 2022

Preprint

View full text Add to dashboard Cite

Vision-based bronchoscopy (VB) models require the registration of the virtual lung model with the frames from the videobronchoscopy to provide an effective guidance during the biopsy. The registration can be achieved by either tracking the position and orientation of the bronchoscopy camera, or by calibrating its deviation from the pose (position and orientation) simulated in the virtual lung model. Recent advances in neural networks and temporal image processing have provided new opportunities for guided bronchoscopy. However, such progress has been hindered by the lack of comparative experimental conditions.In the present paper we share a novel synthetic dataset allowing for a fair comparison of methods. Moreover, this paper investigates several neural network architectures for the learning of temporal information at different levels of subject personalization. In order to improve orientation measurement, we also present a standartized comparison framework and a novel metric for camera orientation learning. Results on the dataset show that the proposed metric and architectures, as well as the standardized conditions, provide notable improvements to current state of the art camera pose estimation in videobronchoscopy.

show abstract

“…In the preparations for planning a surgical path, verifying the location of a lesion, etc., it is an essential tool; in operations such as bronchoscopy, which is the procedure for the inspection and retrieval of diagnostic samples for lung-related surgeries, it is even more so. In Reference [ 1 ], the authors propose a novel registration method to match real bronchoscopy images with virtual bronchoscope images from a 3D bronchial tree model built using computed tomography (CT) image stacks in order to obtain the current 3D position of the bronchoscope in the airways. This method represents a combination of a novel position-tracking method using the current frames from the bronchoscope and the verification of the position of the real bronchoscope image against an image extracted from the 3D model using an adaptive-network-based fuzzy inference system (ANFIS)-based image matching method.…”

mentioning

confidence: 99%