Light Field Image Coding Using High-Order Intrablock Prediction

Monteiro, Ricardo J. S.; Nunes, Paulo; Rodrigues, Nuno M. M.; Faria, Sérgio M. M. de

doi:10.1109/jstsp.2017.2721358

Cited by 38 publications

(21 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [226], a high order compensated prediction is proposed to exploit the fact that neighbor micro-images represent a portion of the scene captured from slightly different perspectives which may not be explored with a translational compensated prediction. For this, geometric transformations with up to 8 degrees of freedom (namely, affine, bilinear and projective) are used to map perspective changes from the block being coded to the causal search window shown in Fig.…”

Section: ) Spatial Compensated Predictionmentioning

confidence: 99%

Dense Light Field Coding: A Survey

2020

View full text Add to dashboard Cite

Light Field (LF) imaging is a promising solution for providing more immersive and closer to reality multimedia experiences to end-users with unprecedented creative freedom and flexibility for applications in different areas, such as virtual and augmented reality. Due to the recent technological advances in optics, sensor manufacturing and available transmission bandwidth, as well as the investment of many tech giants in this area, it is expected that soon many LF transmission systems will be available to both consumers and professionals. Recognizing this, novel standardization initiatives have recently emerged in both the Joint Photographic Experts Group (JPEG) and the Moving Picture Experts Group (MPEG), triggering the discussion on the deployment of LF coding solutions to efficiently handle the massive amount of data involved in such systems. Since then, the topic of LF content coding has become a booming research area, attracting the attention of many researchers worldwide. In this context, this paper provides a comprehensive survey of the most relevant LF coding solutions proposed in the literature, focusing on angularly dense LFs. Special attention is placed on a thorough description of the different LF coding methods and on the main concepts related to this relevant area. Moreover, comprehensive insights are presented into open research challenges and future research directions for LF coding. INDEX TERMS Camera array, image compression, light field, plenoptic, video compression.

show abstract

Section: ) Spatial Compensated Predictionmentioning

confidence: 99%

Dense Light Field Coding: A Survey

2020

View full text Add to dashboard Cite

show abstract

“…These perspective changes require geometric transformations (GT) with more (up to 8) DoF. A method that uses a high order prediction approach was added to a HEVC framework in [27]. Additionally, in [28], an alternative non-local spatial prediction method has been investigated, relying on a prediction mode based on locally linear embedding (LLE) integrated in HEVC.…”

Section: B Mi-based Related Workmentioning

confidence: 99%

“…Moreover, the computational complexity is improved by using different prediction modes for each specific area of the lenslet LF image, i.e., content-based prediction. Coding efficiency is comparable to high order prediction method described in [27] however, no comparisons were performed against SAI-based related work.…”

Section: B Mi-based Related Workmentioning

confidence: 99%

Light Field Image Coding Based on Hybrid Data Representation

et al. 2020

Self Cite

View full text Add to dashboard Cite

This paper proposes a novel efficient light field coding approach based on a hybrid data representation. Current state-of-the-art light field coding solutions either operate on micro-images or sub-aperture images. Consequently, the intrinsic redundancy that exists in light field images is not fully exploited, as is demonstrated. This novel hybrid data representation approach allows to simultaneously exploit four types of redundancies: i) sub-aperture image intra spatial redundancy, ii) sub-aperture image inter-view redundancy, iii) intra-micro-image redundancy, and iv) inter-micro-image redundancy between neighboring micro-images. The proposed light field coding solution allows flexibility for several types of baselines, by adaptively exploiting the most predominant type of redundancy on a coding block basis. To demonstrate the efficiency of using a hybrid representation, this paper proposes a set of efficient pixel prediction methods combined with a pseudo-video sequence coding approach, based on the HEVC standard. Experimental results show consistent average bitrate savings when the proposed codec is compared to relevant state-ofthe-art benchmarks. For lenslet light field content, the proposed coding algorithm outperforms the HEVCbased pseudo-video sequence coding benchmark by an average bitrate savings of 23%. It is shown for the same light field content that the proposed solution outperforms JPEG Pleno verification models MuLE and WaSP, as these codecs are only able to achieve 11% and −14% bitrate savings over the same HEVC-based benchmark, respectively. The performance of the proposed coding approach is also validated for light fields with wider baselines, captured with high-density camera arrays, being able to outperform both the HEVCbased benchmark, as well as MuLE and WaSP.INDEX TERMS Light field representation, light field image coding, HEVC, pseudo-video sequence, spatial pixel prediction, least squares prediction.

show abstract

“…Zhong et al [15] proposed an L1-optimized prediction algorithm that linearly predicts the micro-lens images based on the neighboring reconstructed ones. Monteiro et al [16] proposed a method that relies on a two-stage block-wise high order prediction model, where each image block is intra predicted from a reference in the causal area of the image. However, these methods based on raw LFI cannot exploit the spatial correlation well due to the spatial structure of micro-lens image.…”

Section: Related Work a Lfi Compressionmentioning

confidence: 99%

“…Raw LFI based compression methods [8]- [16] usually directly compress the raw form LFI using image coding tools or intra coding mode of video coding tools with additional predictive strategies. However, the special structure of raw form LFI (shown in Fig.…”

Section: Introductionmentioning

confidence: 99%

Light Field Image Compression via CNN-Based EPI Super-Resolution and Decoder-Side Quality Enhancement

Zhao

Yang

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Because of the capacity of capturing both the spatial and angular information of the light rays simultaneously, light field images (LFIs) contain richer scene information compared with conventional images, but at the cost of huge volume. This paper proposes a novel LFI sparse compression framework driven by convolutional neural network (CNN). The epipolar plane image (EPI) super-resolution is for compensating the information loss caused by sparse sampling and the decoder-side sub-aperture images (SAIs) quality enhancement is for compensating the information loss caused by lossy compression. Specifically, we choose those SAIs both in odd rows and odd columns as our key SAIs and compress them using standard video encoder. For those non-key SAIs, we predict them using decompressed key SAIs by taking advantage of the special structure of EPI. The low-resolution EPIs generated from the sparse SAIs are super-resolved by a CNN and the outputs, high-resolution EPIs, are used to rebuild the dense SAIs. Moreover, in order to improve the quality of the predicted SAI, we add decoder-side quality enhancement before prediction. We propose a multi-scale dense residual network (MSDRN) to implement both EPI super-resolution and quality enhancement. Transfer learning strategies are used to improve the training performance of quality enhancement. The experimental results show the superior performance of the proposed framework over existing methods in terms of rate-distortion performance.INDEX TERMS Light field image, sparse compression, EPI super-resolution, quality enhancement, convolutional neural network.

show abstract

Light Field Image Coding Using High-Order Intrablock Prediction

Cited by 38 publications

References 34 publications

Dense Light Field Coding: A Survey

Dense Light Field Coding: A Survey

Light Field Image Coding Based on Hybrid Data Representation

Light Field Image Compression via CNN-Based EPI Super-Resolution and Decoder-Side Quality Enhancement

Contact Info

Product

Resources

About