Light field compression using depth image based view synthesis

Jiang, Xiaoran; Pendu, Mikaël Le; Guillemot, Christine

doi:10.1109/icmew.2017.8026313

Cited by 50 publications

(43 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…View synthesis based predictive coding has also been investigated in [24] where the authors use a linear approximation computed with Matching Pursuit View synthesis based predictive coding is another research direction followed in [24] where the authors use a linear approximation computed with Matching Pursuit for disparity based view prediction. The authors in [25] and [26] use instead a the convolutional neural network (CNN) architecture proposed in [27] for view synthesis and prediction. The prediction residue is then coded using HEVC [25], or using local residue transforms (SA-DCT) and coding [26].…”

Section: Light Field Compressionmentioning

confidence: 99%

Prediction and Sampling With Local Graph Transforms for Quasi-Lossless Light Field Compression

Rizkallah

Maugey

Guillemot

2020

IEEE Trans. on Image Process.

Self Cite

View full text Add to dashboard Cite

Graph-based transforms have been shown to be powerful tools in terms of image energy compaction. However, when the support increases to best capture signal dependencies, the computation of the basis functions becomes rapidly untractable. This problem is in particular compelling for high dimensional imaging data such as light fields. The use of local transforms with limited supports is a way to cope with this computational difficulty. Unfortunately, the locality of the support may not allow us to fully exploit long term signal dependencies present in both the spatial and angular dimensions in the case of light fields. This paper describes sampling and prediction schemes with local graph-based transforms enabling to efficiently compact the signal energy and exploit dependencies beyond the local graph support. The proposed approach is investigated and is shown to be very efficient in the context of spatio-angular transforms for quasi-lossless compression of light fields.

show abstract

Section: Light Field Compressionmentioning

confidence: 99%

Prediction and Sampling With Local Graph Transforms for Quasi-Lossless Light Field Compression

Rizkallah

Maugey

Guillemot

2020

IEEE Trans. on Image Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…1) as the set of anchor light field views. Arguably, these distinct views on the extreme corners of a densely sampled light field contain all color and geometric information, from which the whole light field can be reconstructed [16]. Disparity is estimated between pairs of anchor views.…”

Section: Finetuned Flownet 20 For Stereomentioning

confidence: 99%

A Learning Based Depth Estimation Framework for 4D Densely and Sparsely Sampled Light Fields

Jiang

Shi

Guillemot

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

This paper proposes a learning based solution to disparity (depth) estimation for either densely or sparsely sampled light fields. Disparity between stereo pairs among a sparse subset of anchor views is first estimated by a fine-tuned FlowNet 2.0 network adapted to disparity prediction task. These coarse estimates are fused by exploiting the photo-consistency warping error, and refined by a Multi-view Stereo Refinement Network (MSRNet). The propagation of disparity from anchor viewpoints towards other viewpoints is performed by an occlusion-aware soft 3D reconstruction method. The experiments show that, both for dense and sparse light fields, our algorithm outperforms significantly the state-of-the-art algorithms, especially for subpixel accuracy.

show abstract

“…Stereo specific compression techniques exist, and range from using a Markov random field (MRF) for disparity prediction to separate transforms for residual images [40,14,48,3,34,42]. Also closely related is light field image compression, where learning-based view synthesis techniques are used to take advantage of the vast redundancy between the subaperture images [20,21].…”

Section: Multi-view Compressionmentioning

confidence: 99%

DSIC: Deep Stereo Image Compression

Liu¹,

Wang

Urtasun³

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

In this paper we tackle the problem of stereo image compression, and leverage the fact that the two images have overlapping fields of view to further compress the representations. Our approach leverages state-of-the-art singleimage compression autoencoders and enhances the compression with novel parametric skip functions to feed fully differentiable, disparity-warped features at all levels to the encoder/decoder of the second image. Moreover, we model the probabilistic dependence between the image codes using a conditional entropy model. Our experiments show an impressive 30 -50% reduction in the second image bitrate at low bitrates compared to deep single-image compression, and a 10 -20% reduction at higher bitrates.

show abstract

Light field compression using depth image based view synthesis

Cited by 50 publications

References 23 publications

Prediction and Sampling With Local Graph Transforms for Quasi-Lossless Light Field Compression

Prediction and Sampling With Local Graph Transforms for Quasi-Lossless Light Field Compression

A Learning Based Depth Estimation Framework for 4D Densely and Sparsely Sampled Light Fields

DSIC: Deep Stereo Image Compression

Contact Info

Product

Resources

About