Monocular Depth Estimation Using Whole Strip Masking and Reliability-Based Refinement

Heo, Minhyeok; Lee, Jaehan; Kim, Kyung Rae; Kim, Han Ul; Kim, Chang-Su

doi:10.1007/978-3-030-01225-0_3

Cited by 39 publications

(25 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Depending on DCN, numerous algorithms are developed, including supervised methods [12], [13], [35], [36] [37], [38], [39] semi-supervised methods [40] or unsupervised methods [41], [42], [43], [44], [45], [46]. Others tried to improve the estimated details further by appending a conditional random field (CRF) [47], [48], [49], [50], [51], [52] and multi-task correlation with joint training [38], [53], [54], [55]. However, the affinity for measuring the coherence of neighboring pixels is commonly manual-designed based on color similarity or intervening contour [56] with RBF kernel [52], [53].…”

Section: Related Workmentioning

confidence: 99%

Learning Depth with Convolutional Spatial Propagation Network

Cheng

Wang

Yang

2020

IEEE Trans. Pattern Anal. Mach. Intell.

246

189

View full text Add to dashboard Cite

In this paper, we propose the convolutional spatial propagation network (CSPN) and demonstrate its effectiveness for various depth estimation tasks. CSPN is a simple and efficient linear propagation model, where the propagation is performed with a manner of recurrent convolutional operations, in which the affinity among neighboring pixels is learned through a deep convolutional neural network (CNN). Compare to the previous state-of-the-art (SOTA) linear propagation model, i.e., spatial propagation networks (SPN), CSPN is 2 to 5× faster in practice. We concatenate CSPN and its variants to SOTA depth estimation networks, which significantly improve the depth accuracy. Specifically, we apply CSPN to two depth estimation problems: depth completion and stereo matching, in which we design modules which adapts the original 2D CSPN to embed sparse depth samples during the propagation, operate with 3D convolution and be synergistic with spatial pyramid pooling. In our experiments, we show that all these modules contribute to the final performance. For the task of depth completion, our method reduce the depth error over 30% in the NYU v2 and KITTI datasets. For the task of stereo matching, our method currently ranks 1st on both the KITTI Stereo 2012 and 2015 benchmarks. The code of CSPN will be released at https://github.com/XinJCheng/CSPN.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning Depth with Convolutional Spatial Propagation Network

Cheng

Wang

Yang

2020

IEEE Trans. Pattern Anal. Mach. Intell.

246

189

View full text Add to dashboard Cite

show abstract

“…Depth map refinement is often treated as a post-processing step, using CRFs [52,57,16,43]: an initial depth prediction is regularized based on pixelwise and pairwise energy terms depending on various guidance signals. These methods now underperform state-of-the-art deep-learning-based methods without refinement [20,58] while being more computationally expensive.…”

Section: Related Workmentioning

confidence: 99%

Pixel-Pair Occlusion Relationship Map (P2ORM): Formulation, Inference and Application

Qiu

Xiao

Wang

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We formalize concepts around geometric occlusion in 2D images (i.e., ignoring semantics), and propose a novel unified formulation of both occlusion boundaries and occlusion orientations via a pixel-pair occlusion relation. The former provides a way to generate large-scale accurate occlusion datasets while, based on the latter, we propose a novel method for task-independent pixel-level occlusion relationship estimation from single images. Experiments on a variety of datasets demonstrate that our method outperforms existing ones on this task. To further illustrate the value of our formulation, we also propose a new depth map refinement method that consistently improve the performance of stateof-the-art monocular depth estimation methods.

show abstract

“…D. Eigen et al used a single multiscale CNN to learn deep features [21]. CNN can be extended with harmonizing overcomplete local predictions [22], multiple candidates in the frequency domain [23], and the whole strip masking module [24]. Nevertheless, these works do not consider the relation between two regional features.…”

Section: Related Workmentioning

confidence: 99%

“…Furtherly, DenseASPP connects more atrous layers with dense connections to obtain larger receptive fields. In our network, we use five atrous convolutional layers in a cascade fashion in DenseASPP, where the dilation rate of each layer increases layer by layer (3,6,12,18,24). We feed DenseASPP with a 64-dimensional feature map 0 y , which is transformed from a 2048-dimensional feature map Re 101 snet y  of ResNet-101 by an FC layer.…”

Section: ) Atrous Convolutionmentioning

confidence: 99%

Monocular Depth Prediction With Residual DenseASPP Network

Zhang

Xie

2020

IEEE Access

View full text Add to dashboard Cite

Monocular depth estimation is an ill-posed problem because infinite 3D scenes can be projected to the same 2D scenes. Most recent methods focus on image-level information from deep convolutional neural networks, while training them may suffer from slow convergence and accuracy degeneration, especially for deeper network and more feature channels. Based on an encoder-decoder framework, we propose a novel Residual DenseASPP Network. In our Residual DenseASPP network, we define features as low/mid/high vision features and use two-kinds of skip connection to learn useful features with certain layers, where feature concentration in the dense block is used to generate more features in the same layer, and feature summation in the residual block is used to increase backward gradient. The experimental results show that high vision features require more channels by feature concentration, while low/mid vision features need better convergence by feature summation. Experiments show that our proposed approach achieves state-of-the-art performance on both NYUv2 and Make3D datasets.

show abstract

Monocular Depth Estimation Using Whole Strip Masking and Reliability-Based Refinement

Cited by 39 publications

References 27 publications

Learning Depth with Convolutional Spatial Propagation Network

Learning Depth with Convolutional Spatial Propagation Network

Pixel-Pair Occlusion Relationship Map (P2ORM): Formulation, Inference and Application

Monocular Depth Prediction With Residual DenseASPP Network

Contact Info

Product

Resources

About