A 3D reconstruction based on an unsupervised domain adaptive for binocular endoscopy

Zhang, Guo; Huang, Zhiwei; Lin, Jinzhao; Li, Zhangyong; Cao, Enling; Pang, Yu; Sun, Weiwei

doi:10.3389/fphys.2022.994343

Cited by 4 publications

(1 citation statement)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, there is no publicly available binocular dataset in the field of MIS. To overcome this problem, researchers have adopted traditional scene datasets [ 13 ], self-supervised convolutional neural networks [ 14 ], and simulation models rendered using Blender [ 15 , 16 ], etc., but the results differ from those of traditional scenes.…”

Section: Introductionmentioning

confidence: 99%

Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM

Huo

Zhou

Yuan

et al. 2023

Sensors

View full text Add to dashboard Cite

Binocular endoscopy is gradually becoming the future of minimally invasive surgery (MIS) thanks to the development of stereo vision. However, some problems still exist, such as the low reconstruction accuracy, small surgical field, and low computational efficiency. To solve these problems, we designed a framework for real-time dense reconstruction in binocular endoscopy scenes. First, we obtained the initial disparity map using an SGBM algorithm and proposed the disparity confidence map as a dataset to provide StereoNet training. Then, based on the depth map predicted by StereoNet, the corresponding left image of each depth map was input into the Oriented Fast and Brief-Simultaneous Localization and Mapping (ORB-SLAM) framework using an RGB-D camera to realize the real-time dense reconstruction of the binocular endoscopy scene. The proposed algorithm was verified in the stomach phantom and a real pig stomach. Compared with the ground truth, the proposed algorithm’s RMSE is 1.620 mm, and the number of effective points in the point cloud is 834,650, which is a significant improvement in the mapping ability compared with binocular SLAM and ensures the real-time performance of the algorithm while performing dense reconstruction. The effectiveness of the proposed algorithm is verified.

show abstract

Section: Introductionmentioning

confidence: 99%

Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM

Huo

Zhou

Yuan

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

Unsupervised Neural Network-Based Image Stitching Method for Bladder Endoscopy

Ye,

Shao,

Zhu

2024

Preprint

View full text Add to dashboard Cite

Bladder endoscopy enables the observation of intravesical lesion characteristics, making it an essential tool in urology. Image stitching techniques are commonly employed to expand the field of view of bladder endoscopy. Traditional image stitching methods rely on feature matching. In recent years, deep-learning techniques have garnered significant attention in the field of computer vision. However, the commonly employed supervised learning approaches often require a substantial amount of labeled data, which can be challenging to acquire, especially in the context of medical data. Both feature-based and unreliable supervised methods for cystoscopy image stitching are limited by their quality and the robustness of image stitching. This study proposes an unsupervised neural network-based image stitching method for bladder endoscopy that comprises two modules: an unsupervised alignment network and an unsupervised fusion network. In the unsupervised alignment network, we employed feature convolution, regression networks, and linear transformations to align images. In the unsupervised fusion network, we achieved image fusion from features to pixel by simultaneously eliminating artifacts and enhancing the resolution. Experiments demonstrated our method's consistent stitching success rate of 98.11% and robust image stitching accuracy at various resolutions. Our method eliminates sutures and flocculent debris from cystoscopy images, presenting good image smoothness while preserving rich textural features. Moreover, our method could successfully stitch challenging scenes such as dim and blurry scenes. Our application of unsupervised deep learning methods in the field of cystoscopy image stitching was successfully validated, laying the foundation for real-time panoramic stitching of bladder endoscopic video images. This advancement provides opportunities for the future development of computer-vision-assisted diagnostic systems for bladder cavities.

show abstract

A Self-Supervised Network-Based Smoke Removal and Depth Estimation for Monocular Endoscopic Videos

Zhang,

Gao,

Meng

et al. 2024

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

A 3D reconstruction based on an unsupervised domain adaptive for binocular endoscopy

Cited by 4 publications

References 54 publications

Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM

Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM

Unsupervised Neural Network-Based Image Stitching Method for Bladder Endoscopy

A Self-Supervised Network-Based Smoke Removal and Depth Estimation for Monocular Endoscopic Videos

Contact Info

Product

Resources

About