Depth Estimation from Monocular Image and Coarse Depth Points based on Conditional GAN

Abstract

Abstract. Depth estimation has achieved considerable success with the development of the depth sensor devices and deep learning method. However, depth estimation from monocular RGB-based image will increase ambiguity and is prone to error. In this paper, we present a novel approach to produce dense depth map from a single image coupled with coarse point-cloud samples. Our approach learns to fit the distribution of the depth map from source data using conditional adversarial networks and convert the sparse point clouds to dense maps. Our experiments show that the use of the conditional adversarial networks can add full image information to the predicted depth maps and the effectiveness of our approach to predict depth in NYU-Depthv2 indoor dataset.

Learning Depth for Scene Reconstruction Using an Encoder-Decoder Model

“…Multimodal data-based depth estimation commonly uses inputs containing two or three modalities of data [7], [8], [35]- [37]. The method [7] converted depth estimation into distance prediction between reference and true depth maps, performing more effectively than the depth prediction [35].…”

“…Wang et al [36] inferred depth by iteratively changing intermediate representation in pre-trained depth estimation models. Li et al [37] employed depth samples and RGB images to estimate depth. Sparse depth-based depth prediction also used deep learning models [3], [8], [38] to predict depth.…”

“…1) A few valid depth points are uniformly and randomly sampled from depth images on training and testing datasets, respectively. For a fair comparison, the number of valid depth samples is set to the same as that of research [8], [37].…”

“…The probability of depth samples at each position is about identical in a depth map because we sample depth points uniformly and randomly. The number of valid depth samples is set to the same as that of work [8], [37], [38] for a fair comparison. In each depth image, the largest number of valid depth samples is 200, which accounts for 0.04% and 0.06% of the total on the KITTI and NYU-Depth-v2 images, respectively.…”

“…To evaluate DEM quantitatively, we use the error metrics which are also adopted by studies [1], [4]- [6], [7], [8], [15], [16], [24]- [26], [27], [28], [35], [37]- [39], [43]. The error metrics consider global statistics between a ground-truth depth image Y containing N depth pixels and a corresponding predicted depth map P consisting of N depth pixels.…”