Depth Coefficients for Depth Completion

Imran, Saif; Long, Y. F.; Liu, Xiaoming; Morris, Daniel

doi:10.1109/cvpr.2019.01273

Cited by 98 publications

(40 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are other ideas that also have been tried to solve this task. Imran et al [28] proposed a new depth representation named Depth Coefficients. Chen et al [6] designed a post-process module to further improve the model performance.…”

Section: Related Workmentioning

confidence: 99%

A Surface Geometry Model for LiDAR Depth Completion

Zhao

Bai

Zhang

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

LiDAR depth completion is a task that predicts depth values for every pixel on the corresponding camera frame although only sparse LiDAR points are available. Most of the existing state-of-the-art solutions are based on deep neural networks, which need a large amount of data and heavy computations for training the models. In this letter, a novel nonlearning depth completion method is proposed by exploiting the local surface geometry that is enhanced by an outlier removal algorithm. The proposed surface geometry model is inspired by the observation that most pixels with unknown depth have a nearby LiDAR point. Therefore, it is assumed those pixels share the same surface with the nearest LiDAR point, and their respective depth can be estimated as the nearest LiDAR depth value plus a residual error. The residual error is calculated by using a derived equation with several physical parameters as input, including the known camera intrinsic parameters, estimated normal vector, and offset distance on the image plane. The proposed method is further enhanced by an outlier removal algorithm that is designed to remove incorrectly mapped LiDAR points from occluded regions. On KITTI dataset, the proposed solution achieves the best error performance among all existing non-learning methods and is comparable to the best self-supervised learning method and some supervised learning methods. Moreover, since removing outlier points coming from occluded regions is a commonly existing problem in LiDAR-Camera systems, the proposed outlier removal algorithm is a general preprocessing step that is applicable to many robotic systems with both camera and LiDAR sensors.

show abstract

Section: Related Workmentioning

confidence: 99%

A Surface Geometry Model for LiDAR Depth Completion

Zhao

Bai

Zhang

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…Conf-Net [15] generates high-confidence point cloud by predicting dense depth error map and filtering out low-confidence points. DC-Coef [16] transforms continuous depth regression into predicting discrete depth bins and applies cross-entropy loss to improve the quality of the point cloud. PwP [17] generates the normal view via principal component analysis (PCA) on a set of neighboring 3D points from sparse ground truth.…”

Section: Related Workmentioning

confidence: 99%

DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network

Xiang

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Depth Completion can produce a dense depth map from a sparse input and provide a more complete 3D description of the environment. Despite great progress made in depth completion, the sparsity of the input and low density of the ground truth still make this problem challenging. In this work, we propose DenseLiDAR, a novel real-time pseudodepth guided depth completion neural network. We exploit dense pseudo-depth map obtained from simple morphological operations to guide the network in three aspects: (1) Constructing a residual structure for the output; (2) Rectifying the sparse input data; (3) Providing dense structural loss for training the network. Thanks to these novel designs, higher performance of the output could be achieved. In addition, two new metrics for better evaluating the quality of the predicted depth map are also presented. Extensive experiments on KITTI depth completion benchmark suggest that our model is able to achieve the state-of-the-art performance at the highest frame rate of 50Hz. The predicted dense depth is further evaluated by several downstream robotic perception or positioning tasks. For the task of 3D object detection, 3˜5 percent performance gains on small objects categories are achieved on KITTI 3D object detection dataset. For RGB-D SLAM, higher accuracy on vehicle's trajectory is also obtained in KITTI Odometry dataset. These promising results not only verify the high quality of our depth prediction, but also demonstrate the potential of improving the related downstream tasks by using depth completion results.

show abstract

“…Therefore, the estimated depths for the indoor scene in general could be incompatible to the known ones. The approaches of Lee et al [27] and Imran et al [28] in literature have considered the problem of completing depth images from sparse sampling. The method presented in [27] learned an extrapolation-like mapping to generate a 'complete' depth image from sparse samples.…”

Section: B Related Workmentioning

confidence: 99%

Fast Generation of High Fidelity RGB-D Images by Deep-Learning with Adaptive Convolution

Xian

Zhang

Dai

et al. 2020

Preprint

View full text Add to dashboard Cite

Using the raw data from consumer-level RGB-D cameras as input, we propose a deep-learning based approach to efficiently generate RGB-D images with completed information in high resolution. To process the input images in low resolution with missing regions, new operators for adaptive convolution are introduced in our deep-learning network that consists of three cascaded modules -the completion module, the refinement module and the super-resolution module. The completion module is based on an architecture of encoder-decoder, where the features of input raw RGB-D will be automatically extracted by the encoding layers of a deep neural-network. The decoding layers are applied to reconstruct the completed depth map, which is followed by a refinement module to sharpen the boundary of different regions. For the super-resolution module, we generate RGB-D images in high resolution by multiple layers for feature extraction and a layer for up-sampling. Benefited from the adaptive convolution operators proposed in this paper, our results outperform the existing deep-learning based approaches for RGB-D image complete and super-resolution. As an endto-end approach, high fidelity RGB-D images can be generated efficiently at the rate of 22 frames per second.

show abstract

Depth Coefficients for Depth Completion

Cited by 98 publications

References 34 publications

A Surface Geometry Model for LiDAR Depth Completion

A Surface Geometry Model for LiDAR Depth Completion

DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network

Fast Generation of High Fidelity RGB-D Images by Deep-Learning with Adaptive Convolution

Contact Info

Product

Resources

About