Passive depth estimation using chromatic aberration and a depth from defocus approach

Trouvé, Pauline; Champagnat, Frédéric; Besnerais, Guy Le; Sabater, J.; Avignon, Thierry; Idier, Jérôme

doi:10.1364/ao.52.007152

Cited by 40 publications

(65 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, dead zones can be overcome using several images with various in-focus planes. In a single snapshot context, this can be obtained with unconventional optics such as a plenoptic camera [32] or a lens with chromatic aberration [33,12], but both at the cost of image quality (low resolution or chromatic aberration).…”

Section: Related Workmentioning

confidence: 99%

Deep Depth from Defocus: How Can Defocus Blur Improve 3D Estimation Using Dense Neural Networks?

Carvalho

Saux

Trouvé-Peloux

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Depth estimation is of critical interest for scene understanding and accurate 3D reconstruction. Most recent approaches in depth estimation with deep learning exploit geometrical structures of standard sharp images to predict corresponding depth maps. However, cameras can also produce images with defocus blur depending on the depth of the objects and camera settings. Hence, these features may represent an important hint for learning to predict depth. In this paper, we propose a full system for single-image depth prediction in the wild using depth-fromdefocus and neural networks. We carry out thorough experiments to test deep convolutional networks on real and simulated defocused images using a realistic model of blur variation with respect to depth. We also investigate the influence of blur on depth prediction observing model uncertainty with a Bayesian neural network approach. From these studies, we show that out-of-focus blur greatly improves the depth-prediction network performances. Furthermore, we transfer the ability learned on a synthetic, indoor dataset to real, indoor and outdoor images. For this purpose, we present a new dataset containing real all-focus and defocused images from a Digital Single-Lens Reflex (DSLR) camera, paired with ground truth depth maps obtained with an active 3D sensor for indoor scenes. The proposed approach is successfully validated on both this new dataset and standard ones as NYUv2 or Depth-in-the-Wild. Code and new datasets are available at https:// github.com/ marcelampc/ d3net depth estimation.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Depth from Defocus: How Can Defocus Blur Improve 3D Estimation Using Dense Neural Networks?

Carvalho

Saux

Trouvé-Peloux

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…This simple and generic model, which can be physically interpreted as a 1/f 2 decrease of the scene spectrum, has previously yielded good results in single image blur identification [9,12,22]. In Section 7, we show that it is also valid to predict the DFD performance on real scenes.…”

Section: B Scene Modelmentioning

confidence: 74%

“…An interesting perspective is to apply this model to the design of SIDFD systems with unconventional cameras, for instance cameras having coded apertures, or a chromatic lens [6,11,12].…”

Section: Resultsmentioning

confidence: 99%

Theoretical performance model for single image depth from defocus

et al. 2014

Self Cite

View full text Add to dashboard Cite

“…In essence, the problem of estimating a depth map from pictorial cues alone is ill-posed. Optically encoding depth-dependent scene information has the potential to remove some of the ambiguities inherent in all-in-focus images, for example using (coded) defocus blur [28,26,20,44,1] or chromatic aberrations [43]. However, it is largely unclear how different optical coding strategies compare to one another and what the best strategy for a specific task may be.…”

Section: Introductionmentioning

confidence: 99%

Deep Optics for Monocular Depth Estimation and 3D Object Detection

Chang

Wetzstein

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

169

112

View full text Add to dashboard Cite

Depth estimation and 3D object detection are critical for scene understanding but remain challenging to perform with a single image due to the loss of 3D information during image capture. Recent models using deep neural networks have improved monocular depth estimation performance, but there is still difficulty in predicting absolute depth and generalizing outside a standard dataset. Here we introduce the paradigm of deep optics, i.e. end-to-end design of optics and image processing, to the monocular depth estimation problem, using coded defocus blur as an additional depth cue to be decoded by a neural network. We evaluate several optical coding strategies along with an end-to-end optimization scheme for depth estimation on three datasets, including NYU Depth v2 and KITTI. We find an optimized freeform lens design yields the best results, but chromatic aberration from a singlet lens offers significantly improved performance as well. We build a physical prototype and validate that chromatic aberrations improve depth estimation on real-world results. In addition, we train object detection networks on the KITTI dataset and show that the lens optimized for depth estimation also results in improved 3D object detection performance.

show abstract

Passive depth estimation using chromatic aberration and a depth from defocus approach

Cited by 40 publications

References 22 publications

Deep Depth from Defocus: How Can Defocus Blur Improve 3D Estimation Using Dense Neural Networks?

Deep Depth from Defocus: How Can Defocus Blur Improve 3D Estimation Using Dense Neural Networks?

Theoretical performance model for single image depth from defocus

Deep Optics for Monocular Depth Estimation and 3D Object Detection

Contact Info

Product

Resources

About