Neural EPI-Volume Networks for Shape from Light Field

Heber, Stefan; Yu, Wei; Pock, Thomas

doi:10.1109/iccv.2017.247

Cited by 97 publications

(91 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This method yielded better results on multi-layered scenes. Heber et al [13] proposed an end-to-end deep network architecture consisting of an encoding and a decoding part. Heber and Pock [12] proposed a combination of a CNN and a variational optimization.…”

Section: Related Workmentioning

confidence: 99%

“…There are still issues in the aforementioned learning based methods. Those methods only consider one directional epipolar geometry of light field images in designing the network [12,13], resulting in low reliability of depth predictions. We overcome this problem via a multi-stream network which encodes each epipolar image separately to improve the depth prediction.…”

Section: Related Workmentioning

confidence: 99%

“…Our EPINET shows the best performance in 3 out of 4 measures. Epinetfcn is our EPINET model using the vertical, the horizontal, the left diagonal and the right diagonal viewspoints as input, [12,13], the error metrics are directly referred from [13]. and Epinet-fcn9x9 is a model that uses all 9×9 viewpoints.…”

Section: Quantitative Evaluationmentioning

confidence: 99%

“…As can be seen from the examples, our EPINET reconstructs the smooth surface and Figure 9. Comparison with deep learning-based methods [12,13]. The results for [12,13] are directly referred from [13].…”

Section: Quantitative Evaluationmentioning

confidence: 99%

See 3 more Smart Citations

EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images

Shin

Jeon²,

Yoon³

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

275

303

View full text Add to dashboard Cite

Light field cameras capture both the spatial and the angular properties of light rays in space. Due to its property, one can compute the depth from light fields in uncontrolled lighting environments, which is a big advantage over active sensing devices. Depth computed from light fields can be used for many applications including 3D modelling and refocusing. However, light field images from hand-held cameras have very narrow baselines with noise, making the depth estimation difficult. Many approaches have been proposed to overcome these limitations for the light field depth estimation, but there is a clear trade-off between the accuracy and the speed in these methods. In this paper, we introduce a fast and accurate light field depth estimation method based on a fully-convolutional neural network. Our network is designed by considering the light field geometry and we also overcome the lack of training data by proposing light field specific data augmentation methods. We achieved the top rank in the HCI 4D Light Field Benchmark on most metrics, and we also demonstrate the effectiveness of the proposed method on real-world light-field images.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Quantitative Evaluationmentioning

confidence: 99%

Section: Quantitative Evaluationmentioning

confidence: 99%

See 2 more Smart Citations

EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images

Shin

Jeon²,

Yoon³

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

275

303

View full text Add to dashboard Cite

show abstract

“…In order to overcome those downsides, the same authors improved their work [8] [9] by utilizing a U-Net [17]. The first paper [8] shows visual and quantitative improvement, but suffers from streaking artifacts, adressed in [9].…”

Section: Methods Based On Deep Learningmentioning

confidence: 99%

Learning to Think Outside the Box: Wide-Baseline Light Field Depth Estimation with EPI-Shift

Leistner

Schilling

Mackowiak

et al. 2019

2019 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

We propose a method for depth estimation from light field data, based on a fully convolutional neural network architecture. Our goal is to design a pipeline which achieves highly accurate results for small-and wide-baseline light fields. Since light field training data is scarce, all learningbased approaches use a small receptive field and operate on small disparity ranges. In order to work with wide-baseline light fields, we introduce the idea of EPI-Shift: To virtually shift the light field stack which enables to retain a small receptive field, independent of the disparity range. In this way, our approach "learns to think outside the box of the receptive field". Our network performs joint classification of integer disparities and regression of disparity-offsets. A U-Net component provides excellent long-range smoothing. EPI-Shift considerably outperforms the state-of-theart learning-based approaches and is on par with handcrafted methods. We demonstrate this on a publicly available, synthetic, small-baseline benchmark and on largebaseline real-world recordings.

show abstract

Deep Depth from Focus

Hazırbaş

Soyer

Staab

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Depth from focus (DFF) is one of the classical ill-posed inverse problems in computer vision. Most approaches recover the depth at each pixel based on the focal setting which exhibits maximal sharpness. Yet, it is not obvious how to reliably estimate the sharpness level, particularly in low-textured areas. In this paper, we propose 'Deep Depth From Focus (DDFF)' as the first end-to-end learning approach to this problem. One of the main challenges we face is the hunger for data of deep neural networks. In order to obtain a significant amount of focal stacks with corresponding groundtruth depth, we propose to leverage a light-field camera with a co-calibrated RGB-D sensor. This allows us to digitally create focal stacks of varying sizes. Compared to existing benchmarks our dataset is 25 times larger, enabling the use of machine learning for this inverse problem. We compare our results with state-ofthe-art DFF methods and we also analyze the effect of several key deep architectural components. These experiments show that our proposed method 'DDFFNet' achieves state-of-the-art performance in all scenes, reducing depth error by more than 75% compared to the classical DFF methods.

show abstract

Neural EPI-Volume Networks for Shape from Light Field

Cited by 97 publications

References 15 publications

EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images

EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images

Learning to Think Outside the Box: Wide-Baseline Light Field Depth Estimation with EPI-Shift

Deep Depth from Focus

Contact Info

Product

Resources

About