Innovative computer vision approach to 3D bladder model reconstruction from flexible cystoscopy

Falcón, Néstor; Ranjbar, Sorush; Cisneros, Exal; Vu, Bao; Schoppe, Austin; Sánchez, Pedro; Jin, Yufang; Ye, Jing Yong; Feng, Yusheng; Kaushik, Dharam; Hood, R. Lyle

doi:10.1117/12.2510725

Cited by 7 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Determination of the clinical readiness of a given reconstruction pipeline requires objective evaluation tools that can assess its reliability and potential to work in a particular clinical scenario or to perform well under a variety of potential clinical scenarios. While 3D reconstruction pipelines have been developed for several clinical applications [10] , [13] – [27] , a robust set of evaluation tools has not been established. The lack of such tools makes it difficult to identify which aspects of a newly developed pipeline should be changed to improve its performance, or to compare different pipelines to determine which is better for a certain clinical application scenario.…”

Section: Introductionmentioning

confidence: 99%

“…Hence, evaluations using these datasets do not generalize well to the clinical domain [32] . As a result, most virtual endoscopy developers perform evaluations using proprietary datasets [10] , [13] – [27] . Not only are these datasets not broadly available, but they also represent only a limited range of clinical scenarios, which masks pipeline generalizability to different scenarios.…”

Section: Introductionmentioning

confidence: 99%

“…To obtain ground truth of organ shape, textures and camera poses, some prior works have used preoperational computed tomography (CT) scans of the organ or laser scans of physical phantoms and camera poses measured by commercial trackers [20] , [22] – [25] , [33] . However, these ground truths do not consider possible tissue deformation, the complexity of which is a major obstacle in the development of a robust 3D reconstruction pipeline for clinical use [27] .…”

Section: Introductionmentioning

confidence: 99%

“…Finally, the metrics often used to evaluate reconstruction pipelines provide only a limited view of the pipeline’s performance [12] , [20] , [22] – [25] , [33] , making it hard to assess whether new pipelines are superior or inferior to existing options. For example, most works report subjective assessment of the 3D model’s visual appearance and/or the quantitative residual distance obtained after aligning the reconstructed 3D model with the ground truth model.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Cost-Efficient Video Synthesis and Evaluation for Development of Virtual 3D Endoscopy

Zhou

Eimen

Seibel

et al. 2021

IEEE J. Transl. Eng. Health Med.

View full text Add to dashboard Cite

Objective: 3D reconstruction of the shape and texture of hollow organs captured by endoscopy is important for the diagnosis and surveillance of early and recurrent cancers. Better evaluation of 3D reconstruction pipelines developed for such applications requires easy access to extensive datasets and associated ground truths, cost-efficient and scalable simulations of a range of possible clinical scenarios, and more reliable and insightful metrics to assess performance. Methods: We present a computer-aided simulation platform for cost-effective synthesis of monocular endoscope videos and corresponding ground truths that mimic a range of potential settings and situations one might encounter during acquisition of clinical endoscopy videos. Using cystoscopy of the bladder as model case, we generated an extensive dataset comprising several synthesized videos of a bladder phantom. We then introduce a novel evaluation procedure to reliably assess an individual 3D reconstruction pipeline or to compare different pipelines. Results: To illustrate the use of the proposed platform and evaluation procedure, we use the aforementioned dataset and ground truths to evaluate a proprietary 3D reconstruction pipeline (CYSTO3D) for bladder cystoscopy videos and compared it with a general-purpose 3D reconstruction pipeline (COLMAP). The evaluation results provide insight into the suggested clinical acquisition protocol and several potential areas for refinement of the pipeline to improve future performance. Conclusion: Our work proposes an endoscope video synthesis and reconstruction evaluation toolset and presents experimental results that illustrate usage of the toolset to efficiently assess performance and reveal possible problems of any given 3D reconstruction pipeline, to compare different pipelines, and to provide technically or clinically actionable insights.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Cost-Efficient Video Synthesis and Evaluation for Development of Virtual 3D Endoscopy

Zhou

Eimen

Seibel

et al. 2021

IEEE J. Transl. Eng. Health Med.

View full text Add to dashboard Cite

show abstract

Towards the Development of a Digital Twin for Endoscopic Medical Device Testing

Kalozoumis

Marino

Carniel

et al. 2022

Studies in Systems, Decision and Control

View full text Add to dashboard Cite

3-D and 2-D reconstruction of bladders for the assessment of inter-session detection of tissue changes: a proof of concept

et al. 2023

View full text Add to dashboard Cite

Purpose Abnormalities in the bladder wall require careful investigation regarding type, spatial position and invasiveness. Construction of a 3-D model of the bladder is helpful to ensure adequate coverage of the scanning procedure, quantitative comparison of bladder wall textures between successive sessions and finding back previously discovered abnormalities. Methods Videos of both an in vivo bladder and a textured bladder phantom were acquired. Structure-from-motion and bundle adjustment algorithms were used to construct a 3-D point cloud, approximate it by a surface mesh, texture it with the back-projected camera frames and draw the corresponding 2-D atlas. Reconstructions of successive sessions were compared; those of the bladder phantom were co-registered, transformed using 3-D thin plate splines and post-processed to highlight significant changes in texture. Results The reconstruction algorithms of the presented workflow were able to construct 3-D models and corresponding 2-D atlas of both the in vivo bladder and the bladder phantom. For the in vivo bladder the portion of the reconstructed surface area was 58% and 79% for the pre- and post-operative scan, respectively. For the bladder phantom the full surface was reconstructed and the mean reprojection error was 0.081 mm (range 0–0.79 mm). In inter-session comparison the changes in texture were correctly indicated for all six locations. Conclusion The proposed proof of concept was able to perform 3-D and 2-D reconstruction of an in vivo bladder wall based on a set of monocular images. In a phantom study the computer vision algorithms were also effective in co-registering reconstructions of successive sessions and highlighting texture changes between sessions. These techniques may be useful for detecting, monitoring and revisiting suspicious lesions.

show abstract

Innovative computer vision approach to 3D bladder model reconstruction from flexible cystoscopy

Cited by 7 publications

References 17 publications

Cost-Efficient Video Synthesis and Evaluation for Development of Virtual 3D Endoscopy

Cost-Efficient Video Synthesis and Evaluation for Development of Virtual 3D Endoscopy

Towards the Development of a Digital Twin for Endoscopic Medical Device Testing

3-D and 2-D reconstruction of bladders for the assessment of inter-session detection of tissue changes: a proof of concept

Contact Info

Product

Resources

About