An Efficient Random Access Light Field Video Compression Utilizing Diagonal Inter-View Prediction

Mehajabin, Nusrat; Luo, Sichen Roger; Yu, Hao; Khoury, Joseph; Kaur, Jashandeep; Pourazad, Mahsa T.

doi:10.1109/icip.2019.8803668

Cited by 16 publications

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We consider the average and the maximum number of reference frames required to be decoded for a view as the average and worst-case random access complexity respectively. We evaluate the random access complexity of the coding schemes in the experiments by using the following criteria [52]:…”

Section: Random Access Complexity Analysismentioning

confidence: 99%

Pre-Demosaic Graph-Based Light Field Image Compression

Chao

Hong

Cheung

et al. 2022

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

A plenoptic light field (LF) camera places an array of microlenses in front of an image sensor in order to separately capture different directional rays arriving at an image pixel. Using a conventional Bayer pattern, data captured at each pixel is a single color component (R, G or B). The sensed data then undergoes demosaicking (interpolation of RGB components per pixel) and conversion to an array of sub-aperture images (SAIs). In this paper, we propose a new LF image coding scheme based on graph lifting transform (GLT), where the acquired sensor data are coded in the original captured form without pre-processing. Specifically, we directly map raw sensed color data to the SAIs, resulting in sparsely distributed color pixels on 2D grids, and perform demosaicking at the receiver after decoding. To exploit spatial correlation among the sparse pixels, we propose a novel intra-prediction scheme, where the prediction kernel is determined according to the local gradient estimated from already coded neighboring pixel blocks. We then connect the pixels by forming a graph, modeling the prediction residuals statistically as a Gaussian Markov Random Field (GMRF). The optimal edge weights are computed via a graph learning method using a set of training SAIs. The residual data is encoded via low-complexity GLT. Experiments show that at high PSNRsimportant for archiving and instant storage scenarios-our method outperformed significantly a conventional light field image coding scheme with demosaicking followed by High Efficiency Video Coding (HEVC).

show abstract

Section: Random Access Complexity Analysismentioning

confidence: 99%

Pre-Demosaic Graph-Based Light Field Image Compression

Chao

Hong

Cheung

et al. 2022

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

show abstract

“…Alternatively, in [27] and [28] SAIs are interpreted as a HDCA signal and encoded with MV-HEVC where two-dimensional weighted prediction and rate allocation is available. In [40], a MV-HEVC based coding solution was proposed, that allows diagonal viewpoint prediction instead of exclusively the horizontal and vertical viewpoint prediction. Experimental results show that allowing diagonal viewpoint prediction provides a good compromise between coding efficiency and viewpoint random access when compared to algorithms that are exclusively based on horizontal and vertical viewpoint prediction.…”

Section: Sai-based Related Workmentioning

confidence: 99%

Light Field Image Coding Based on Hybrid Data Representation

et al. 2020

View full text Add to dashboard Cite

This paper proposes a novel efficient light field coding approach based on a hybrid data representation. Current state-of-the-art light field coding solutions either operate on micro-images or sub-aperture images. Consequently, the intrinsic redundancy that exists in light field images is not fully exploited, as is demonstrated. This novel hybrid data representation approach allows to simultaneously exploit four types of redundancies: i) sub-aperture image intra spatial redundancy, ii) sub-aperture image inter-view redundancy, iii) intra-micro-image redundancy, and iv) inter-micro-image redundancy between neighboring micro-images. The proposed light field coding solution allows flexibility for several types of baselines, by adaptively exploiting the most predominant type of redundancy on a coding block basis. To demonstrate the efficiency of using a hybrid representation, this paper proposes a set of efficient pixel prediction methods combined with a pseudo-video sequence coding approach, based on the HEVC standard. Experimental results show consistent average bitrate savings when the proposed codec is compared to relevant state-ofthe-art benchmarks. For lenslet light field content, the proposed coding algorithm outperforms the HEVCbased pseudo-video sequence coding benchmark by an average bitrate savings of 23%. It is shown for the same light field content that the proposed solution outperforms JPEG Pleno verification models MuLE and WaSP, as these codecs are only able to achieve 11% and −14% bitrate savings over the same HEVC-based benchmark, respectively. The performance of the proposed coding approach is also validated for light fields with wider baselines, captured with high-density camera arrays, being able to outperform both the HEVCbased benchmark, as well as MuLE and WaSP.INDEX TERMS Light field representation, light field image coding, HEVC, pseudo-video sequence, spatial pixel prediction, least squares prediction.

show abstract