Dynamic Fusion Network For Light Field Depth Estimation

Abstract

Focus-based methods have shown promising results for the task of depth estimation. However, most existing focus-based depth estimation approaches depend on maximal sharpness of the focal stack. Out-of-focus information in the focal stack poses challenges for this task. In this paper, we propose a dynamically multi-modal learning strategy which incorporates RGB data and the focal stack in our framework. Our goal is to deeply excavate the spatial correlation in the focal stack by designing the spatial-correlation perception module and dynamically fuse multi-modal information between RGB data and the focal stack in a adaptive way by designing the multi-modal dynamic fusion module. The success of our method is demonstrated by achieving the state-ofthe-art performance on two datasets. Furthermore, we test our network on a set of different focused images generated by a smart-phone camera to prove that the proposed method not only broke the limitation of only using light field data, but also open a path toward practical applications of depth estimation on common consumer level cameras data. The code is available: https://github.com/OIPLab-DUT/Light-Field-for-Depth-Estimation.

RGB-D-Fusion: Image Conditioned Depth Diffusion of Humanoid Subjects

“…Other deep learning-based approaches focus on multiple sensor modalities to estimate the depth of the scene. [67] use LiDAR point clouds in combination with stereo images, [68] use monocular RGB images combined with sparse LiDAR point clouds, [69] input monocular RGB combined with a depth map and [70] combine a single RGB image with a focal stack.…”

Light field salient object detection: A review and benchmark

“…As a result, it can be viewed as an array of images captured by a grid of cameras. Compared to RGB images captured by a regular camera or depth maps acquired by a depth sensor, the light field data acquired by a plenoptic camera records more comprehensive and complete information about natural scenes, covering, for example, depth information [38][39][40][41][42][43][44], focusness cues [5,42], and angular changes [42,45]. Therefore, light field data can benefit SOD in a number of ways.…”