Depth from Gradients in Dense Light Fields for Object Reconstruction

Yücer, Kaan; Kim, Chang-Il; Sorkine-Hornung, Alexander; Sorkine‐Hornung, Olga

doi:10.1109/3dv.2016.33

Cited by 21 publications

(17 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Neri et al [25] presents multi-resolution method, based on local minimization of the maximum likelihood functional. The approach in [33] utilizes the patch-based local gradient information for depth calculation with further propagation. Navarro and Buades [24] propose a combination of two stereo non-dense methods, which in conjunction with interpolation gives a dense depth map.…”

Section: Related Work 21 Light Field Analysismentioning

confidence: 99%

Fast and Efficient Depth Map Estimation from Light Fields

Anisimov

Stricker

2017

2017 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

The paper presents an algorithm for depth map estimation from the light field images in relatively small amount of time, using only single thread on CPU. The proposed method improves existing principle of line fitting in 4dimensional light field space. Line fitting is based on color values comparison using kernel density estimation. Our method utilizes result of Semi-Global Matching (SGM) with Census transform-based matching cost as a border initialization for line fitting. It provides a significant reduction of computations needed to find the best depth match. With the suggested evaluation metric we show that proposed method is applicable for efficient depth map estimation while preserving low computational time compared to others.

show abstract

Section: Related Work 21 Light Field Analysismentioning

confidence: 99%

Fast and Efficient Depth Map Estimation from Light Fields

Anisimov

Stricker

2017

2017 International Conference on 3D Vision (3DV)

View full text Add to dashboard Cite

show abstract

“…Unlike conventional segmentation approaches, such as co-segmentation and multiview segmentation, views in a light field are much more correlated (i.e., EPI volume), and thus can help in ensuring labeling consistency. However, due to the large number of views in a light field (usually a dozen [157] to a hundred views [158], [159]), segmentation and matting using light field data take abundant computing resources and running time during the energy minimization, which restricts their further promotion. Campbell et al presented multiple-view segmentation approaches using graph-cuts in voxel space [161] and image space [162].…”

Section: Segmentation and Mattingmentioning

confidence: 99%

“…Wanner et al [157] presented a variational framework for multi-label segmentation on the ray space of 4D light fields. Yücer et al [158], [159] proposed a segmentation approach that uses unstructured 3D light fields captured by a hand-held video camera (see Fig. 16(a)).…”

Section: Segmentation and Mattingmentioning

confidence: 99%

“…Further examples of applications using light fields. (a) Object segmentation proposed by Yücer et al [158], [159], where the likelihood of edges belonging to the foreground was estimated using the gradients in the 3D light field volume. The left column shows a sub-aperture image and an EPI of the light field, the middle column shows the segmentation results for them, and the right column shows the 3D segmentation result.…”

Section: Detection Classification and Recognitionmentioning

confidence: 99%

See 1 more Smart Citation

Light Field Image Processing: An Overview

Masiá

Jarabo

et al. 2017

IEEE J. Sel. Top. Signal Process.

492

215

View full text Add to dashboard Cite

Light field imaging has emerged as a technology allowing to capture richer visual information from our world. As opposed to traditional photography, which captures a 2D projection of the light in the scene integrating the angular domain, light fields collect radiance from rays in all directions, demultiplexing the angular information lost in conventional photography. On the one hand, this higher-dimensional representation of visual data offers powerful capabilities for scene understanding, and substantially improves the performance of traditional computer vision problems such as depth sensing, post-capture refocusing, segmentation, video stabilization, material classification, etc. On the other hand, the high-dimensionality of light fields also brings up new challenges in terms of data capture, data compression, content editing and display. Taking these two elements together, research in light field image processing has become increasingly popular in the computer vision, computer graphics and signal processing communities. In this article, we present a comprehensive overview and discussion of research in this field over the past 20 years. We focus on all aspects of light field image processing, including basic light field representation and theory, acquisition, super-resolution, depth estimation, compression, editing, processing algorithms for light field display, and computer vision applications of light field data. Index Terms-Light field imaging, light field processing.

show abstract

“…Furthermore, the photos have often been taken at different points in time, making 3D reconstruction problematic as the illumination can differ significantly and objects may have moved in the scene [39]. Finally, another interesting capture strategy is unstructured video capture, where a user records video while moving the camera to capture multiple viewpoints [40,41,42]. In this thesis, we employ unstructured capture strategies that can easily be performed by just one camera operator, and also provide footage suitable for VR experiences with 360°coverage of the scene.…”

Section: Unstructured Capturementioning

confidence: 99%

Viewpoint-Free Photography for Virtual Reality

Hedman¹

2020

Real VR – Immersive Digital Reality

View full text Add to dashboard Cite

Viewpoint-free photography, i.e., interactively controlling the viewpoint of a photograph after capture, is a standing challenge. In this thesis, we investigate algorithms to enable viewpoint-free photography for virtual reality (VR) from casual capture, i.e., from footage easily captured with consumer cameras. We build on an extensive body of work in image-based rendering (IBR). Given images of an object or scene, IBR methods aim to predict the appearance of an image taken from a novel perspective. Most IBR methods focus on full or near-interpolation, where the output viewpoints either lie directly between captured images, or nearby. These methods are not suitable for VR, where the user has significant range of motion and can look in all directions. Thus, it is essential to create viewpoint-free photos with a wide field-of-view and sufficient positional freedom to cover the range of motion a user might experience in VR. We focus on two VR experiences: 1) Seated VR experiences, where the user can lean in different directions. This simplifies the problem, as the scene is only observed from a small range of viewpoints. Thus, we focus on easy capture, showing how to turn panorama-style capture into 3D photos, a simple representation for viewpoint-free photos, and also how to speed up processing so users can see the final result on-site. 2) Room-scale VR experiences, where the user can explore vastly different perspectives. This is challenging: More input footage is needed, maintaining real-time display rates becomes difficult, view-dependent appearance and object backsides need to be modelled, all while preventing noticeable mistakes. We address these challenges by: (1) creating refined geometry for each input photograph, (2) using a fast tiled rendering algorithm to achieve real-time display rates, and (3) using a convolutional neural network to hide visual mistakes during compositing.

show abstract

Depth from Gradients in Dense Light Fields for Object Reconstruction

Cited by 21 publications

References 33 publications

Fast and Efficient Depth Map Estimation from Light Fields

Fast and Efficient Depth Map Estimation from Light Fields

Light Field Image Processing: An Overview

Viewpoint-Free Photography for Virtual Reality

Contact Info

Product

Resources

About