Real-Time Dense Depth Estimation Using Semantically-Guided LIDAR Data Propagation and Motion Stereo

Hirata, Atsuki; Ishikawa, Ryoichi; Roxas, Menandro; Oishi, Takeshi

doi:10.1109/lra.2019.2927126

Cited by 11 publications

(17 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the second evaluation, we examined the robustness of the proposed method against LiDARcamera extrinsic calibration errors. In this experiment, we used the KITTI [24] and Komaba datasets [38] with added calibration errors. In all evaluations, we implemented SSM and B-ADT aided smoothing on GPU by CUDA, and used RANSAC plane segmentation from PCL library [50] for the ground detection.…”

Section: Discussionmentioning

confidence: 99%

“…In this method, the smoothness term is weighted as per texture derivatives; however, the results suffer from surface overflattening. To address this issue, Ferstl et al formalized depth completion into ADT-aided and TGV-regularized energy minimization [3]; and their method has been successfully used to smooth and optimize depth maps in more recent methods [37], [38]. Recently, Yao et al proposed B-ADT to achieve depth completion to preserve discontinuity between different objects [8].…”

Section: B Single-image-aided Depth Completionmentioning

confidence: 99%

“…The Komaba dataset was introduced in the literature [38] and has been used in a previous study [8]. The figure of an example frame of the Komaba dataset is in the supplementary material.…”

Section: ) Komaba Datasetmentioning

confidence: 99%

See 2 more Smart Citations

Non-Learning Stereo-Aided Depth Completion Under Mis-Projection via Selective Stereo Matching

Yao¹,

Ishikawa

Ando

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

We propose a non-learning depth completion method for a sparse depth map captured using a light detection and ranging (LiDAR) sensor guided by a pair of stereo images. Generally, conventional stereo-aided depth completion methods have two limiations. (i) They assume the given sparse depth map is accurately aligned to the input image, whereas the alignment is difficult to achieve in practice.(ii) They have limited accuracy in the long range because the depth is estimated by pixel disparity. To solve the abovementioned limitations, we propose selective stereo matching (SSM) that searches the most appropriate depth value for each image pixel from its neighborly projected LiDAR points based on an energy minimization framework. This depth selection approach can handle any type of mis-projection. Moreover, SSM has an advantage in terms of long-range depth accuracy because it directly uses the LiDAR measurement rather than the depth acquired from the stereo. SSM is a discrete process; thus, we apply variational smoothing with binary anisotropic diffusion tensor (B-ADT) to generate a continuous depth map while preserving depth discontinuity across object boundaries. Experimentally, compared with the previous state-of-the-art stereo-aided depth completion, the proposed method reduced the mean absolute error (MAE) of the depth estimation to 0.65 times and demonstrated approximately twice more accurate estimation in the long range. Moreover, under various LiDAR-camera calibration errors, the proposed method reduced the depth estimation MAE to 0.34-0.93 times from previous depth completion methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: B Single-image-aided Depth Completionmentioning

confidence: 99%

See 1 more Smart Citation

Non-Learning Stereo-Aided Depth Completion Under Mis-Projection via Selective Stereo Matching

Yao¹,

Ishikawa

Ando

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, the method does not apply to scenes without camera and LIDAR motion. Schneider et al proposed a method that pre-trained semantic segmentation for guided upsampling of sparse depth maps [6], and Hirata et al used pre-trained semantic segmentation and motion stereo for a similar purpose [7]. In principle, supervised methods apply only to scenes of the same domain as the training.…”

Section: A Supervised Methodsmentioning

confidence: 99%

“…To accommodate this, Ferstl et al formalized depth completion into ADT-aided TGV regularized energy minimization [2]. Their method was successfully used to smooth and optimize depth maps in more recent methods [7], [12]. Nevertheless, since ADT does not eliminate smoothness along occlusion boundaries, it produces surface-like artifacts between foreground and background objects.…”

Section: B Unsupervised Methodsmentioning

confidence: 99%

Discontinuous and Smooth Depth Completion With Binary Anisotropic Diffusion Tensor

Yao

Roxas

Ishikawa

et al. 2020

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

We propose an unsupervised real-time dense depth completion from a sparse depth map guided by a single image. Our method generates a smooth depth map while preserving discontinuity between different objects. Our key idea is a Binary Anisotropic Diffusion Tensor (B-ADT) which can completely eliminate smoothness constraint at intended positions and directions by applying it to variational regularization. We also propose an Image-guided Nearest Neighbor Search (IGNNS) to derive a piecewise constant depth map which is used for B-ADT derivation and in the data term of the variational energy. Our experiments show that our method can outperform previous unsupervised and semi-supervised depth completion methods in terms of accuracy. Moreover, since our resulting depth map preserves the discontinuity between objects, the result can be converted to a visually plausible point cloud. This is remarkable since previous methods generate unnatural surface-like artifacts between discontinuous objects.

show abstract

RGB Guided ToF Imaging System: A Survey of Deep Learning-Based Methods

Qiao,

Poggi,

Deng

et al. 2024

Int J Comput Vis

View full text Add to dashboard Cite

Real-Time Dense Depth Estimation Using Semantically-Guided LIDAR Data Propagation and Motion Stereo

Cited by 11 publications

References 25 publications

Non-Learning Stereo-Aided Depth Completion Under Mis-Projection via Selective Stereo Matching

Non-Learning Stereo-Aided Depth Completion Under Mis-Projection via Selective Stereo Matching

Discontinuous and Smooth Depth Completion With Binary Anisotropic Diffusion Tensor

RGB Guided ToF Imaging System: A Survey of Deep Learning-Based Methods

Contact Info

Product

Resources

About