Light field image coding with jointly estimated self-similarity bi-prediction

Conti, Caroline; Nunes, Paulo; Soares, Luís Ducla

doi:10.1016/j.image.2017.10.006

Cited by 19 publications

(13 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in this case, to guarantee that the two prediction signals come from two different MIs, the search area is separated into two non-overlapping parts. In [218], [219], the authors propose to extend their previous work in [212] with a bi-SS prediction. Different from the solution in [206], [217], the predictor block is generated from a combination of two candidate blocks jointly estimated from the same search area.…”

Section: ) Spatial Compensated Predictionmentioning

confidence: 99%

Dense Light Field Coding: A Survey

2020

Self Cite

View full text Add to dashboard Cite

Light Field (LF) imaging is a promising solution for providing more immersive and closer to reality multimedia experiences to end-users with unprecedented creative freedom and flexibility for applications in different areas, such as virtual and augmented reality. Due to the recent technological advances in optics, sensor manufacturing and available transmission bandwidth, as well as the investment of many tech giants in this area, it is expected that soon many LF transmission systems will be available to both consumers and professionals. Recognizing this, novel standardization initiatives have recently emerged in both the Joint Photographic Experts Group (JPEG) and the Moving Picture Experts Group (MPEG), triggering the discussion on the deployment of LF coding solutions to efficiently handle the massive amount of data involved in such systems. Since then, the topic of LF content coding has become a booming research area, attracting the attention of many researchers worldwide. In this context, this paper provides a comprehensive survey of the most relevant LF coding solutions proposed in the literature, focusing on angularly dense LFs. Special attention is placed on a thorough description of the different LF coding methods and on the main concepts related to this relevant area. Moreover, comprehensive insights are presented into open research challenges and future research directions for LF coding. INDEX TERMS Camera array, image compression, light field, plenoptic, video compression.

show abstract

Section: ) Spatial Compensated Predictionmentioning

confidence: 99%

Dense Light Field Coding: A Survey

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Each enhancement layer is encoded with the proposed LF enhancement layer codec seen in Fig. 4, which is based on the HEVC architecture and may explore spatial and inter-layer redundancy through self-similarity (SS) [11] and inter-layer predictions (modules shown in blue); a detailed description of the modules used for inter-layer prediction is provided in [13].…”

Section: A Fovs-lfc Architecture With Roi Supportmentioning

confidence: 99%

“…The enhancement layer was encoded with the solution in Fig. 4 with only SS prediction [11]. In the following, these will be referred to as "Scalable ROI Coding".…”

Section: A Test Conditionsmentioning

confidence: 99%

See 1 more Smart Citation

Scalable Light Field Coding with Support for Region of Interest Enhancement

Conti

Soares

Nunes

2018

2018 26th European Signal Processing Conference (EUSIPCO)

Self Cite

View full text Add to dashboard Cite

Light field imaging based on microlens arraysa.k.a. holoscopic, plenoptic, and integral imaging-has currently risen up as a feasible and prospective technology for future image and video applications. However, deploying actual light field applications will require identifying more powerful representation and coding solutions that support emerging manipulation and interaction functionalities. In this context, this paper proposes a novel scalable coding approach that supports a new type of scalability, referred to as Field of View (FOV) scalability, in which enhancement layers can correspond to regions of interest (ROI). The proposed scalable coding approach comprises a base layer compliant with the High Efficiency Video Coding (HEVC) standard, complemented by one or more enhancement layers that progressively allow richer versions of the same light field content in terms of content manipulation and interaction possibilities, for the whole scene or just for a given ROI. Experimental results show the advantages of the proposed scalable coding approach with ROI support to cater for users with different preferences/requirements in terms of interaction functionalities. Keywords-light field, field of view scalability, region of interest, image compression, HEVC This work was supported by FCT (Fundação para a Ciência e a Tecnologia, Portugal) under the UID/EEA/50008/2013 project.

show abstract

“…The authors in [12] presented a low order prediction (LOP) tool to exploit neighboring MI redundancy, based on a block matching approach. This LOP tool was later on extended [13] with a bi-directional search algorithm. Alternatively, a prediction tool based on template matching and Locally Linear Embedding (LLE) was proposed in [14], [16], allowing to outperform the LOP tool.…”

Section: Introductionmentioning

confidence: 99%

Light field image coding: objective performance assessment of Lenslet and 4D LF data representations

Monteiro

Rodrigues

Faria

et al. 2018

Applications of Digital Image Processing XLI

Self Cite

View full text Add to dashboard Cite

State-of-the-art light field (LF) image coding solutions, usually, rely in one of two LF data representation formats: Lenslet or 4D LF. While the Lenslet data representation is a more compact version of the LF, it requires additional camera metadata and processing steps prior to image rendering. On the contrary, 4D LF data, consisting of a stack of sub-aperture images, provides a more redundant representation requiring, however, minimal side information, thus facilitating image rendering. Recently, JPEG Pleno guidelines on objective evaluation of LF image coding defined a processing chain that allows to compare different 4D LF data codecs, aiming to facilitate codec assessment and benchmark. Thus, any codec that does not rely on the 4D LF representation needs to undergo additional processing steps to generate an output comparable to a reference 4D LF image. These additional processing steps may have impact on the quality of the reconstructed LF image, especially if color subsampling format and bit depth conversions have been performed. Consequently, the influence of these conversions needs to be carefully assessed as it may have a significant impact on a comparison between different LF codecs. Very few in-depth comparisons on the effects of using existing LF representation have been reported. Therefore, using the guidelines from JPEG Pleno, this paper presents an exhaustive comparative analysis of these two LF data representation formats in terms of LF image coding efficiency, considering different color subsampling formats and bit depths. These comparisons are performed by testing different processing chains to encode and decode the LF images. Experimental results have shown that, in terms of coding efficiency for different color subsampling formats, the Lenslet LF data representation is more efficient when using YUV 4:4:4 with 10 bit/sample, while the 4D LF data representation is more efficient when using YUV 4:2:0 with 8 bit/sample. The "best" LF data representation, in terms of coding efficiency, depends on several factors which are extensively analyzed in this paper, such as the objective metric that is used for comparison (e.g., average PSNR-Y or average PNSR-YUV), the type of LF content, as well as the color format. The maximum objective quality is also determined, by evaluating the influence of each block from each processing chain in the objective quality of the reconstructed LF image. Experimental results show that, when the 4D LF data representation is not used the maximum achieved objective quality is lower than 50 dB, in terms of average PSNR-YUV.

show abstract

Light field image coding with jointly estimated self-similarity bi-prediction

Abstract: Light field image coding with jointly estimated self-similarity bi-prediction. Signal Processing: Image Communication. 60, 144-159, which has been published in final form at

Cited by 19 publications

References 35 publications

Dense Light Field Coding: A Survey

Dense Light Field Coding: A Survey

Scalable Light Field Coding with Support for Region of Interest Enhancement

Light field image coding: objective performance assessment of Lenslet and 4D LF data representations

Contact Info

Product

Resources

About