Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields

Abstract

Abstract-In this article, we tackle the problem of depth estimation from single monocular images. Compared with depth estimation using multiple images such as stereo depth perception, depth from monocular images is much more challenging. Prior work typically focuses on exploiting geometric priors or additional sources of information, most using hand-crafted features. Recently, there is mounting evidence that features from deep convolutional neural networks (CNN) set new records for various vision applications. On the other hand, considering the continuous characteristic of the depth values, depth estimation can be naturally formulated as a continuous conditional random field (CRF) learning problem. Therefore, here we present a deep convolutional neural field model for estimating depths from single monocular images, aiming to jointly explore the capacity of deep CNN and continuous CRF. In particular, we propose a deep structured learning scheme which learns the unary and pairwise potentials of continuous CRF in a unified deep CNN framework. We then further propose an equally effective model based on fully convolutional networks and a novel superpixel pooling method, which is about 10 times faster, to speedup the patch-wise convolutions in the deep model. With this more efficient model, we are able to design deeper networks to pursue better performance. Our proposed method can be used for depth estimation of general scenes with no geometric priors nor any extra information injected. In our case, the integral of the partition function can be calculated in a closed form such that we can exactly solve the log-likelihood maximization. Moreover, solving the inference problem for predicting depths of a test image is highly efficient as closed-form solutions exist. Experiments on both indoor and outdoor scene datasets demonstrate that the proposed method outperforms state-of-the-art depth estimation approaches.

Single Image Depth Estimation With Integration of Parametric Learning and Non-Parametric Sampling

“…In [20], they replace the fully connected CRF into mordern domain transform. Likewise semantic segmentation, depth estimation methods based on FCN concentrate how to refine depth boundaries using CRF [21], or annotations of relative depth [22].…”

Inaccuracy-Tolerant Sparse-to-Dense Depth Propagation for Semiautomatic 2D-to-3D Conversion

“…Automatic methods infer depth information in image/video by exploiting different depth perception cues such as motion, occlusion, vanishing points, defocus, and so on. Recently, with the popularity of deep learning, many neural networks have been proposed for automatic depth estimation [7][8][9]. However, existing automatic methods can generally provide a limited 3D effect due to ambiguities between depth and perception cues [2].…”

Prevention and management of intraoperative crisis in VATS and open chest surgery: how to avoid emergency conversion

“…Adaption to this lack of depth perception occurs with experience through a number of conscious and subconscious mechanisms and techniques including object interposition, relative scales of motion between objects, alteration between near and far views, alteration between views through different ports, shadowing, assessment of texture variation and gradients, and knowledge of known object sizes. Combined, these mechanisms can closely approximate a three-dimensional view to the experienced surgeon (11)(12)(13)(14). Measures that may predict successful outcomes in cases of bleeding or other complications include the designation of a dedicated thoracic surgical team and the availability of appropriate instrumentation and equipment at the time of emergency (15)(16)(17)(18)(19)(20).…”