Joint Dense 3D Reconstruction Method for Endoscopic Images of Weak Texture Scenes

Abstract

Endoscopic inspection is an important non-destructive testing method. Traditional 3D endoscopic reconstruction methods, such as polarization reconstruction and shading reconstruction, have the drawbacks of not being able to determine the actual size and positional information of the object. The stereo vision method is limited by its own operating principles and has the issue sparse reconstructed point clouds. These drawbacks greatly limit the applications of the endoscope. Therefore, this work proposes a joint dense 3D reconstruction method for endoscopic imaging of weak texture scenes. This method uses the shading reconstruction normal to correct the polarization reconstruction normal, then uses coordinate conversion and point cloud fusion to convert the polarization and shading 3D reconstruction results from the pixel coordinate system to the world coordinate system. It combined the reconstruction results from the polarization, shading and stereo vision in the world coordinate system, and the fusion coefficients are obtained by solving the minimum error model; then, a complete and detailed 3D reconstruction surface was obtained in the world coordinate system. This method could avoid the difficulty of obtaining real coordinates for 3D reconstruction of polarization and shading and the issue of the sparse point cloud afforded by stereo vision reconstruction for weak texture scenes. Finally, a dense point cloud corresponding to each pixel in the world coordinate system could be obtained. The combined dense 3D reconstruction method had an average error of <1% for length measurement of a 3D curve, which is of high significance for industrial endoscopic inspection.

Polarization Snapshot Imaging Spectrometer for Infrared Range

“…Finally, the image on the roof-shaped stepped micromirror enters the rear imaging system through the beam splitter and is finally imaged on the detector. From this, we can obtain the polarization interference image array for the target object and the three-dimensional polarization interference intensity datasets I 0 (x,y,∆(m,n)), I 45 (x,y,∆(m,n)), I 90 (x,y,∆(m,n)) and I 135 (x,y,∆(m,n)) for the four polarization directions of the target by segmenting and registering the polarization interference image array [19,20]. ∆(m,n) is the optical path difference modulated by the phase modulation unit (m,n): ∆(m,n) = 2(Nn−m)d. Here, m and n are the position coordinates of the corresponding phase modulation unit, taking the center of the whole modulation unit as the zero point and increasing accordingly in the horizontal and vertical directions.…”

Research on 3D Reconstruction of Binocular Vision Based on Thermal Infrared

“…Binocular stereo vision is a prominent research focus in the field of machine vision [1]. Over the years, it has played a vital role in various domains, including 3D reconstruction [2], aerospace [3], and unmanned driving [4]. Among existing depth-sensing technologies, stereo vision stands out as the only passive method, offering advantages such as low cost, ease of implementation, and the ability to range under non-contact conditions.…”

Light Switching Microprojector Allows Endoscopic In Vivo 3D Imaging of Gastrointestinal Abnormalities

“…[ 20 ] However, the biometric errors mainly depend on the operational experience due to the quantitative information deficiencies of conventional stereoscopic methods. [ 25 ] In addition, the insertion of an additional camera module substantially increases the catheter diameter of clinical endoscopes. [ 26 ] In contrast, structured illumination utilizes a projector module near a single endoscopic camera to extract a lesion's quantitative depth profile by measuring the distortion of the projected light pattern on a surface.…”