Compressed Animated Light Fields with Real-Time View-Dependent Reconstruction

Koniaris, Charalampos; Kosek, Maggie; Sinclair, D.; Mitchell, Kim

doi:10.1109/tvcg.2018.2818156

Cited by 17 publications

(7 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, the method of light field reconstruction quality optimization is derived by spectrum analysis of the light field for non-Lambertian surface scenes. In [5], Koniaris et al proposed animated light fields with realtime view correlation reconstruction by converting offline rendered movie content into a novel immersive representation. In [31], Marquez et al proposed a compressed spectrum analysis method and a light field imaging architecture based on the principle of a compressed imaging framework to capture the representation of multiplexed multidimensional information.…”

Section: A Reconstruction Using Spectral Analysismentioning

confidence: 99%

See 1 more Smart Citation

Multiview Reconstruction Using Phase Retrieval of Light Field Signals

He¹,

Zhang²,

Zhu

2022

IEEE Access

View full text Add to dashboard Cite

Light field rendering is an effective multi-view reconstruction technology. The current multiview reconstruction methods still has the problem of low view synthesis quality in complex scenes, such as occlusion and non Lambertian surfaces. In this article, we consider undistorted multiview reconstruction and develop a phase retrieval method for light field (PRLF) signals to improve the rendering quality. Our approach utilizes the phase spectrum in both the time domain and frequency domain of the discrete light field because the phase of the light field carries more scene information. The phase corrections are "light field truncation" in the time domain and "phase replacement" in the frequency domain. The iterative algorithm used provides high-quality estimated results for the relative phase of the light field. By the estimated phase results, we derive a multiview reconstruction method to improve the rendering quality. Finally, to verify the claimed performance, we also compare the PRLF with the most advanced light field reconstruction algorithms. The experimental results show that the proposed PRLF outperforms other known reconstruction schemes.INDEX TERMS Light field rendering, phase retrieval, novel view reconstruction, rendering quality I. INTRODUCTION

show abstract

Section: A Reconstruction Using Spectral Analysismentioning

confidence: 99%

“…The reconstruction problem of a novel view given only a set of captured multiview images, as shown in Fig. 1, appears in a wide range of computational imaging and virtual reality applications [3], [4], [5], [6]. Nonetheless, for the reconstruction of LFR technology [7], the solutions described previously still have some limitations.…”

mentioning

confidence: 99%

Multiview Reconstruction Using Phase Retrieval of Light Field Signals

He¹,

Zhang²,

Zhu

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…2 Complexity e regression model based on cascading shapes first creates a rough face shape and then gradually optimizes the rough shape to the most suitable point by a multistage regression. Because it can be used to train large-scale data, it is easy to operate and it is more convenient to replace different features, so the algorithm is popular as soon as it is proposed [19]. But it also faces some drawbacks, such as slow positioning speed and low positioning accuracy when the posture is complex.…”

Section: Related Workmentioning

confidence: 99%

3D Face Modeling Algorithm for Film and Television Animation Based on Lightweight Convolutional Neural Network

Cheng

Peng

et al. 2021

Complexity

View full text Add to dashboard Cite

Through the analysis of facial feature extraction technology, this paper designs a lightweight convolutional neural network (LW-CNN). The LW-CNN model adopts a separable convolution structure, which can propose more accurate features with fewer parameters and can extract 3D feature points of a human face. In order to enhance the accuracy of feature extraction, a face detection method based on the inverted triangle structure is used to detect the face frame of the images in the training set before the model extracts the features. Aiming at the problem that the feature extraction algorithm based on the difference criterion cannot effectively extract the discriminative information, the Generalized Multiple Maximum Dispersion Difference Criterion (GMMSD) and the corresponding feature extraction algorithm are proposed. The algorithm uses the difference criterion instead of the entropy criterion to avoid the “small sample” problem, and the use of QR decomposition can extract more effective discriminative features for facial recognition, while also reducing the computational complexity of feature extraction. Compared with traditional feature extraction methods, GMMSD avoids the problem of “small samples” and does not require preprocessing steps on the samples; it uses QR decomposition to extract features from the original samples and retains the distribution characteristics of the original samples. According to different change matrices, GMMSD can evolve into different feature extraction algorithms, which shows the generalized characteristics of GMMSD. Experiments show that GMMSD can effectively extract facial identification features and improve the accuracy of facial recognition.

show abstract

“…Since the introduction of light fields to the computer graphics community by [LH96] and [GGSC96], a large number of methods have been proposed, ranging from light field acquisition and compression to processing and view synthesis, for an overview, see [WMJ*17], Light fields can be captured in different ways, including micro‐lens array (MLA) setups [NLB*05, AW92], coded masks [VRA*07, WIH13], objective lens arrays, and gantry and array‐based camera systems [BSW01, KKSM19]. A few designs have been suggested that can support light field video capture.…”

Section: Introductionmentioning

confidence: 99%

Light Field Video Compression and Real Time Rendering

Hajisharif

Miandji

Larsson

et al. 2019

Computer Graphics Forum

View full text Add to dashboard Cite

Light field imaging is rapidly becoming an established method for generating flexible image based description of scene appearances. Compared to classical 2D imaging techniques, the angular information included in light fields enables effects such as post‐capture refocusing and the exploration of the scene from different vantage points. In this paper, we describe a novel GPU pipeline for compression and real‐time rendering of light field videos with full parallax. To achieve this, we employ a dictionary learning approach and train an ensemble of dictionaries capable of efficiently representing light field video data using highly sparse coefficient sets. A novel, key element in our representation is that we simultaneously compress both image data (pixel colors) and the auxiliary information (depth, disparity, or optical flow) required for view interpolation. During playback, the coefficients are streamed to the GPU where the light field and the auxiliary information are reconstructed using the dictionary ensemble and view interpolation is performed. In order to realize the pipeline we present several technical contributions including a denoising scheme enhancing the sparsity in the dataset which enables higher compression ratios, and a novel pruning strategy which reduces the size of the dictionary ensemble and leads to significant reductions in computational complexity during the encoding of a light field. Our approach is independent of the light field parameterization and can be used with data from any light field video capture system. To demonstrate the usefulness of our pipeline, we utilize various publicly available light field video datasets and discuss the medical application of documenting heart surgery.

show abstract

Compressed Animated Light Fields with Real-Time View-Dependent Reconstruction

Cited by 17 publications

References 20 publications

Multiview Reconstruction Using Phase Retrieval of Light Field Signals

Multiview Reconstruction Using Phase Retrieval of Light Field Signals

3D Face Modeling Algorithm for Film and Television Animation Based on Lightweight Convolutional Neural Network

Light Field Video Compression and Real Time Rendering

Contact Info

Product

Resources

About