Learning Depth Completion of Transparent Objects using Augmented Unpaired Data

Erich, Floris; Leme, Bruno; Ando, Noriaki; Hanai, Ryo; Domae, Yukiyasu

doi:10.1109/icra48891.2023.10160619

Cited by 4 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(3) Applications to create a real (stereo) training dataset for deep neural networks using our new TranSpec3D dataset of transparent and specular objects without object painting (cf. [ 36 , 40 , 41 ]).…”

Section: Conclusion Limitations and Future Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…In the field of deep stereo, for example, the generation of real training data with dense ground truth disparities is very complex (costly and time-consuming), especially for visually uncooperative objects in the visible spectral range [ 1 , 36 , 40 , 41 ], e.g., specular, non-reflective, or non-textured surfaces. Here, the painting of uncooperative objects is SOTA [ 36 , 40 , 41 ]. However, in collaboration with the Fraunhofer Institute IOF, we have developed a new method without object painting based on a novel 3D thermal infrared sensing technique by Landmann et al [ 42 ].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Triangle-Mesh-Rasterization-Projection (TMRP): An Algorithm to Project a Point Cloud onto a Consistent, Dense and Accurate 2D Raster Image

Junger

Buch

Notni

2023

Sensors

View full text Add to dashboard Cite

The projection of a point cloud onto a 2D camera image is relevant in the case of various image analysis and enhancement tasks, e.g., (i) in multimodal image processing for data fusion, (ii) in robotic applications and in scene analysis, and (iii) for deep neural networks to generate real datasets with ground truth. The challenges of the current single-shot projection methods, such as simple state-of-the-art projection, conventional, polygon, and deep learning-based upsampling methods or closed source SDK functions of low-cost depth cameras, have been identified. We developed a new way to project point clouds onto a dense, accurate 2D raster image, called Triangle-Mesh-Rasterization-Projection (TMRP). The only gaps that the 2D image still contains with our method are valid gaps that result from the physical limits of the capturing cameras. Dense accuracy is achieved by simultaneously using the 2D neighborhood information (rx,ry) of the 3D coordinates in addition to the points P(X,Y,V). In this way, a fast triangulation interpolation can be performed. The interpolation weights are determined using sub-triangles. Compared to single-shot methods, our algorithm is able to solve the following challenges. This means that: (1) no false gaps or false neighborhoods are generated, (2) the density is XYZ independent, and (3) ambiguities are eliminated. Our TMRP method is also open source, freely available on GitHub, and can be applied to almost any sensor or modality. We also demonstrate the usefulness of our method with four use cases by using the KITTI-2012 dataset or sensors with different modalities. Our goal is to improve recognition tasks and processing optimization in the perception of transparent objects for robotic manufacturing processes.

show abstract

Section: Conclusion Limitations and Future Workmentioning

confidence: 99%