Densely-Sampled Light Field Reconstruction

Vagharshakyan, Suren; Bregović, Robert; Gotchev, Atanas

doi:10.1007/978-3-030-41816-8_3

Cited by 3 publications

(1 citation statement)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, the proposed parallax view generation approach, Parallax-Interpolation Adaptive Separable Convolution (PIASC), leverages a fine-tuning strategy to enhance the convolution kernels of SepConv with a consideration of the motion coherence of static objects in a parallax-view capture system. The PIASC method is evaluated on all the three development datasets of ICME 2018 grand challenge on DSLF reconstruction [VSB+18] and further compared with SepConv. Experimental results demonstrate the effectiveness of the proposed PIASC and its superiority over SepConv for DSLF reconstruction of static scenes.…”

Section: Publication 4 : Parallax View Generation For Static Scenes Umentioning

confidence: 99%

Large-Scale Light Field Capture and Reconstruction

Yuan¹

View full text Add to dashboard Cite

This thesis discusses approaches and techniques to convert Sparsely- Sampled Light Fields (SSLFs) into Densely-Sampled Light Fields (DSLFs), which can be used for visualization on 3DTV and Virtual Reality (VR) de- vices. Exemplarily, a movable 1D large-scale light ﬁeld acquisition system for capturing SSLFs in real-world environments is evaluated. This system consists of 24 sparsely placed RGB cameras and two Kinect V2 sensors. The real-world SSLF data captured with this setup can be leveraged to reconstruct real-world DSLFs. To this end, three challenging problems require to be solved for this system: (i) how to estimate the rigid trans- formation from the coordinate system of a Kinect V2 to the coordinate system of an RGB camera; (ii) how to register the two Kinect V2 sensors with a large displacement; (iii) how to reconstruct a DSLF from a SSLF with moderate and large disparity ranges. To overcome these three challenges, we propose: (i) a novel self- calibration method, which takes advantage of the geometric constraints from the scene and the cameras, for estimating the rigid transformations from the camera coordinate frame of one Kinect V2 to the camera coordi- nate frames of 12-nearest RGB cameras; (ii) a novel coarse-to-ﬁne approach for recovering the rigid transformation from the coordinate system of one Kinect to the coordinate system of the other by means of local color and geometry information; (iii) several novel algorithms that can be categorized into two groups for reconstructing a DSLF from an input SSLF, including novel view synthesis methods, which are inspired by the state-of-the-art video frame interpolation algorithms, and Epipolar-Plane Image (EPI) in- painting methods, which are inspired by the Shearlet Transform (ST)-based DSLF reconstruction approaches.

show abstract

Section: Publication 4 : Parallax View Generation For Static Scenes Umentioning

confidence: 99%

Large-Scale Light Field Capture and Reconstruction

Yuan¹

View full text Add to dashboard Cite

show abstract

Densely-Sampled Light Field Reconstruction

Vagharshakyan

Bregović

Gotchev

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Self-Supervised Light Field Reconstruction Using Shearlet Transform and Cycle Consistency

Gao

Bregović

Gotchev

2020

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

The image-based rendering approach using Shearlet Transform (ST) is one of the state-of-theart Densely-Sampled Light Field (DSLF) reconstruction methods. It reconstructs Epipolar-Plane Images (EPIs) in image domain via an iterative regularization algorithm restoring their coefficients in shearlet domain. Consequently, the ST method tends to be slow because of the time spent on domain transformations for dozens of iterations. To overcome this limitation, this letter proposes a novel selfsupervised DSLF reconstruction method, CycleST, which applies ST and cycle consistency to DSLF reconstruction. Specifically, CycleST is composed of an encoder-decoder network and a residual learning strategy that restore the shearlet coefficients of densely-sampled EPIs using EPI reconstruction and cycle consistency losses. Besides, CycleST is a self-supervised approach that can be trained solely on Sparsely-Sampled Light Fields (SSLFs) with small disparity ranges ( 8 pixels). Experimental results of DSLF reconstruction on SSLFs with large disparity ranges (16 -32 pixels) from two challenging real-world light field datasets demonstrate the effectiveness and efficiency of the proposed CycleST method. Furthermore, CycleST achieves ∼ 9x speedup over ST, at least.

show abstract

Densely-Sampled Light Field Reconstruction

Cited by 3 publications

References 56 publications

Large-Scale Light Field Capture and Reconstruction

Large-Scale Light Field Capture and Reconstruction

Densely-Sampled Light Field Reconstruction

Self-Supervised Light Field Reconstruction Using Shearlet Transform and Cycle Consistency

Contact Info

Product

Resources

About