Deep Depth from Focus

Hazırbaş, Caner; Soyer, Sebastian Georg; Staab, Maximilian Christian; Leal-Taixé, Laura; Cremers, Daniel

doi:10.1007/978-3-030-20893-6_33

Cited by 45 publications

(69 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternative strategy to constraining the geometry of the scene is to vary the camera's focus. Using this "depth from (de)focus" [24] approach, depth can be estimated from focal stacks using classic vision techniques [53] or deep learning approaches [26]. Focus can be made more informative in depth estimation by manually "coding" the aperture of a camera [40], thereby causing the camera's circle of confusion to more explicitly encode scene depth.…”

Section: Related Workmentioning

confidence: 99%

Learning Single Camera Depth Estimation Using Dual-Pixels

Garg

Wadhwa

Ansari

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

114

View full text Add to dashboard Cite

Deep learning techniques have enabled rapid progress in monocular depth estimation, but their quality is limited by the ill-posed nature of the problem and the scarcity of high quality datasets. We estimate depth from a single camera by leveraging the dual-pixel auto-focus hardware that is increasingly common on modern camera sensors. Classic stereo algorithms and prior learning-based depth estimation techniques underperform when applied on this dualpixel data, the former due to too-strong assumptions about RGB image matching, and the latter due to not leveraging the understanding of optics of dual-pixel image formation. To allow learning based methods to work well on dual-pixel imagery, we identify an inherent ambiguity in the depth estimated from dual-pixel cues, and develop an approach to estimate depth up to this ambiguity. Using our approach, existing monocular depth estimation techniques can be effectively applied to dual-pixel data, and much smaller models can be constructed that still infer high quality depth. To demonstrate this, we capture a large dataset of in-the-wild 5-viewpoint RGB images paired with corresponding dualpixel data, and show how view supervision with this data can be used to learn depth up to the unknown ambiguity. On our new task, our model is 30% more accurate than any prior work on learning-based monocular or stereoscopic depth estimation.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning Single Camera Depth Estimation Using Dual-Pixels

Garg

Wadhwa

Ansari

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

114

View full text Add to dashboard Cite

show abstract

“…Second, dead zones can be overcome using several images with various in-focus planes. In a single snapshot context, this can be obtained with unconventional optics such as a plenoptic camera [32] or a lens with chromatic aberration [33,12], but both at the cost of image quality (low resolution or chromatic aberration).…”

Section: Related Workmentioning

confidence: 99%

“…Learning depth from defocus blur. The existence of common datasets for depth estimation [35,1,32], containing pairs of RGB images and corresponding depth maps, facilitates the creation of synthetic defocused images using real camera parameters. Hence, a deep learning approach can be used.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Deep Depth from Defocus: How Can Defocus Blur Improve 3D Estimation Using Dense Neural Networks?

Carvalho

Saux

Trouvé-Peloux

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Depth estimation is of critical interest for scene understanding and accurate 3D reconstruction. Most recent approaches in depth estimation with deep learning exploit geometrical structures of standard sharp images to predict corresponding depth maps. However, cameras can also produce images with defocus blur depending on the depth of the objects and camera settings. Hence, these features may represent an important hint for learning to predict depth. In this paper, we propose a full system for single-image depth prediction in the wild using depth-fromdefocus and neural networks. We carry out thorough experiments to test deep convolutional networks on real and simulated defocused images using a realistic model of blur variation with respect to depth. We also investigate the influence of blur on depth prediction observing model uncertainty with a Bayesian neural network approach. From these studies, we show that out-of-focus blur greatly improves the depth-prediction network performances. Furthermore, we transfer the ability learned on a synthetic, indoor dataset to real, indoor and outdoor images. For this purpose, we present a new dataset containing real all-focus and defocused images from a Digital Single-Lens Reflex (DSLR) camera, paired with ground truth depth maps obtained with an active 3D sensor for indoor scenes. The proposed approach is successfully validated on both this new dataset and standard ones as NYUv2 or Depth-in-the-Wild. Code and new datasets are available at https:// github.com/ marcelampc/ d3net depth estimation.

show abstract

“…Classic methods like Shape from Shading [27,38] and depth from focus [9,32] elaborated physical and mathematical property about light shading and focal setting at each pixel. Recent works extended classic methods by machine learning, like deep depth from focus [12] and deep estimation based on fourier domain analysis [19]. Fully connected convolution (FCN) networks are used to predict depth map [18] or refine coarse-scale depth value [6].…”

Section: Depth Estimationmentioning

confidence: 99%

Indoor Depth Completion with Boundary Consistency and Self-Attention

Huang

Liu

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Depth estimation features are helpful for 3D recognition. Commodity-grade depth cameras are able to capture depth and color image in real-time. However, glossy, transparent or distant surface cannot be scanned properly by the sensor. As a result, enhancement and restoration from sensing depth is an important task. Depth completion aims at filling the holes that sensors fail to detect, which is still a complex task for machine to learn. Traditional hand-tuned methods have reached their limits, while neural network based methods tend to copy and interpolate the output from surrounding depth values. This leads to blurred boundaries, and structures of the depth map are lost. Consequently, our main work is to design an end-to-end network improving completion depth maps while maintaining edge clarity. We utilize self-attention mechanism, previously used in image inpainting fields, to extract more useful information in each layer of convolution so that the complete depth map is enhanced. In addition, we propose boundary consistency concept to enhance the depth map quality and structure. Experimental results validate the effectiveness of our selfattention and boundary consistency schema, which outperforms previous state-of-the-art depth completion work on Matterport3D dataset. Our code is publicly available at

show abstract

Deep Depth from Focus

Cited by 45 publications

References 41 publications

Learning Single Camera Depth Estimation Using Dual-Pixels

Learning Single Camera Depth Estimation Using Dual-Pixels

Deep Depth from Defocus: How Can Defocus Blur Improve 3D Estimation Using Dense Neural Networks?

Indoor Depth Completion with Boundary Consistency and Self-Attention

Contact Info

Product

Resources

About