Generative Localization With Uncertainty Estimation Through Video-CT Data for Bronchoscopic Biopsy

Zhao, Cheng; Shen, Mali; Sun, Li; Yang, Guang‐Zhong

doi:10.1109/lra.2019.2955941

Cited by 25 publications

(11 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our paper also adds value to the learning-based methods used for VNB. Recently, two studies introduced supervised learning-based VNB, validated on phantom videos (Sganga et al, 2019;Zhao et al, 2020). The mean tracking accuracy reported in (Sganga et al, 2019) was 10 mm (if the airway was correctly recognized).…”

Section: Discussionmentioning

confidence: 99%

“…However, there was no accuracy reported for particular bronchi, and their method got lost in the early upper lobe. The median tracking accuracy reported in (Zhao et al, 2020) was 1.64 mm; however, there was no insight on the airway generation in which the accuracy was investigated. These works are far from clinical adoption in terms of training datasets and technical metrics used for validation.…”

Section: Discussionmentioning

confidence: 99%

“…Moreover, their supervised nature requires manually labeled data, which constrains them significantly from further clinical advancement. Note that the approach presented in (Zhao et al, 2020) shares the same unsupervised Cyclic-Gan core as (Shen et al, 2019) and our approach. However, they use it to estimate virtual bronchoscopic images to further train another network for pose estimation in a supervised way.…”

Section: Discussionmentioning

confidence: 99%

“…To avoid constraining our method to supervised training, we decided to proceed with geometry-based ICP camera pose estimation. Also, due to the unsupervised nature of the proposed depth estimation method we have not recorded ground truth poses for our training sequences, which does not allow us to train a pose estimator presented in (Zhao et al, 2020).…”

Section: Discussionmentioning

confidence: 99%

See 3 more Smart Citations

Visually Navigated Bronchoscopy using three cycle-Consistent generative adversarial network for depth estimation

Banach

King

Masaki

et al. 2021

Medical Image Analysis

View full text Add to dashboard Cite

Background-Electromagnetically Navigated Bronchoscopy (ENB) is currently the state-of-the art diagnostic and interventional bronchoscopy. CT-to-body divergence is a critical hurdle in ENB, causing navigation error and ultimately limiting the clinical efficacy of diagnosis and treatment. In this study, Visually Navigated Bronchoscopy (VNB) is proposed to address the aforementioned issue of CT-to-body divergence.Materials and Methods-We extended and validated an unsupervised learning method to generate a depth map directly from bronchoscopic images using a Three Cycle-Consistent Generative Adversarial Network (3cGAN) and registering the depth map to preprocedural CTs. We tested the working hypothesis that the proposed VNB can be integrated to the navigated bronchoscopic system based on 3D Slicer, and accurately register bronchoscopic images to pre-procedural CTs to navigate transbronchial biopsies. The quantitative metrics to asses the hypothesis we set was Absolute Tracking Error (ATE) of the tracking and the Target Registration Error (TRE) of the total navigation system. We validated our method on phantoms produced from the pre-procedural CTs of five patients who underwent ENB and on two ex-vivo pig lung specimens.Results-The ATE using 3cGAN was 6.2 +/− 2.9 [mm]. The ATE of 3cGAN was statistically significantly lower than that of cGAN, particularly in the trachea and lobar bronchus (p < 0.001). The TRE of the proposed method had a range of 11.7 to 40.5 [mm]. The TRE computed by 3cGAN was statistically significantly smaller than those computed by cGAN in two of the five cases enrolled (p < 0.05).Conclusion-VNB, using 3cGAN to generate the depth maps was technically and clinically feasible. While the accuracy of tracking by cGAN was acceptable, the TRE warrants further investigation and improvement.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Visually Navigated Bronchoscopy using three cycle-Consistent generative adversarial network for depth estimation

Banach

King

Masaki

et al. 2021

Medical Image Analysis

View full text Add to dashboard Cite

show abstract

“…With adversarial learning, domain adaptation between the real domain and the synthetic domain is accomplished. Previously, cycleGAN-like architectures has been used to adapt real bronchoscopy images to virtual style images [ 20 ]. Real-to-virtual adaptation was also used for colonoscopy using a generative adversarial network (GAN) architecture [ 21 ].…”

Section: Introductionmentioning

confidence: 99%

Real-to-virtual domain transfer-based depth estimation for real-time 3D annotation in transnasal surgery: a study of annotation accuracy and stability

Tong

Liu

et al. 2021

Int J CARS

View full text Add to dashboard Cite

Purpose Surgical annotation promotes effective communication between medical personnel during surgical procedures. However, existing approaches to 2D annotations are mostly static with respect to a display. In this work, we propose a method to achieve 3D annotations that anchor rigidly and stably to target structures upon camera movement in a transnasal endoscopic surgery setting. Methods This is accomplished through intra-operative endoscope tracking and monocular depth estimation. A virtual endoscopic environment is utilized to train a supervised depth estimation network. An adversarial network transfers the style from the real endoscopic view to a synthetic-like view for input into the depth estimation network, wherein framewise depth can be obtained in real time. Results (1) Accuracy: Framewise depth was predicted from images captured from within a nasal airway phantom and compared with ground truth, achieving a SSIM value of 0.8310 ± 0.0655. (2) Stability: mean absolute error (MAE) between reference and predicted depth of a target point was 1.1330 ± 0.9957 mm. Conclusion Both the accuracy and stability evaluations demonstrated the feasibility and practicality of our proposed method for achieving 3D annotations.

show abstract

A novel intrarenal multimodal 2D/3D registration algorithm and preliminary phantom study

Jin

Zhang

et al. 2023

J Applied Clin Med Phys

View full text Add to dashboard Cite

Retrograde intrarenal surgery (RIRS) is a widely utilized diagnostic and therapeutic tool for multiple upper urinary tract pathologies. The image-guided navigation system can assist the surgeon to perform precise surgery by providing the relative position between the lesion and the instrument after the intraoperative image is registered with the preoperative model. However, due to the structural complexity and diversity of multi-branched organs such as kidneys, bronchi, etc., the consistency of the intensity distribution of virtual and real images will be challenged, which makes the classical pure intensity registration method prone to bias and random results in a wide search domain. In this paper, we propose a structural feature similarity-based method combined with a semantic style transfer network, which significantly improves the registration accuracy when the initial state deviation is obvious. Furthermore, multi-view constraints are introduced to compensate for the collapse of spatial depth information and improve the robustness of the algorithm. Experimental studies were conducted on two models generated from patient data to evaluate the performance of the method and competing algorithms. The proposed method obtains mean target error (mTRE) of 0.971 ± 0.585 mm and 1.266 ± 0.416 mm respectively, with better accuracy and robustness overall. Experimental results demonstrate that the proposed method has the potential to be applied to RIRS and extended to other organs with similar structures.

show abstract

Generative Localization With Uncertainty Estimation Through Video-CT Data for Bronchoscopic Biopsy

Abstract: This is a repository copy of Generative localisation with uncertainty estimation through video-CT data for bronchoscopic biopsy.

Cited by 25 publications

References 21 publications

Visually Navigated Bronchoscopy using three cycle-Consistent generative adversarial network for depth estimation

Visually Navigated Bronchoscopy using three cycle-Consistent generative adversarial network for depth estimation

Real-to-virtual domain transfer-based depth estimation for real-time 3D annotation in transnasal surgery: a study of annotation accuracy and stability

A novel intrarenal multimodal 2D/3D registration algorithm and preliminary phantom study

Contact Info

Product

Resources

About