Specular Reflections Removal of Gastrointestinal Polyps Based on Endoscopic Image

Qian, Kai; Chen, Xu; Feng, Bin; An, Ziheng

doi:10.1109/icsip55141.2022.9886111

Cited by 2 publications

(1 citation statement)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The fourth way of handling specular highlights is to inpaint the affected regions with the underlying texture. Some methods rely on an optimisation formulation that searches for well-matching patches or pixels to the occluded ones in the rest of the image or previous frames and uses it as a replacement (Guo et al, 2016;Shen et al, 2020;Chao et al, 2021;Qian et al, 2022;Nie et al, 2023). In these methods, motion fields are assumed to be homogeneous and thus do not perform well with complex motions.…”

Section: Endoscopic Specular Highlight Processingmentioning

confidence: 99%

A Temporal Learning Approach to Inpainting Endoscopic Specularities and Its effect on Image Correspondence

Daher¹,

Júnior²,

Stoyanov³

2022

Proceedings of the 14th Hamlyn Symposium on Medical Robotics 2022

View full text Add to dashboard Cite

Computer vision has been utilized to analyze mini- mally invasive surgery videos and aid with polyp detec- tion, tool localization, and organ 3D modelling tasks. However, irregular light patterns such as saturation, specular highlights, or extreme contrasts occlude texture and hinder these tasks. In this work, specular highlights were removed and the occluded data was reconstructed. To do that, an unsupervised temporal generative ad- versarial network (GAN) was used to inpaint specular highlights spatially and temporally. Due to the absence of a dataset with ground truth occluded textures, the network was trained on the in-vivo gastric endoscopy dataset (Hyper-Kvasir [1]) with specular highlight masks that were automatically created and processed to act as pseudo ground truths. Ablation studies and direct comparison with other methods were used to show the improved results of our system. In addition, the results on various datasets show the generalizability of our net- work on different environments and procedures as shown in Fig. 1. Finally, experiments also show the positive effect of inpainting on other computer vision tasks under the umbrella of 3D reconstruction and localization in endoscopy including feature matching as well as optical flow and disparity estimation.

show abstract