Pre-Demosaic Graph-Based Light Field Image Compression

Abstract

A plenoptic light field (LF) camera places an array of microlenses in front of an image sensor in order to separately capture different directional rays arriving at an image pixel. Using a conventional Bayer pattern, data captured at each pixel is a single color component (R, G or B). The sensed data then undergoes demosaicking (interpolation of RGB components per pixel) and conversion to an array of sub-aperture images (SAIs). In this paper, we propose a new LF image coding scheme based on graph lifting transform (GLT), where the acquired sensor data are coded in the original captured form without pre-processing. Specifically, we directly map raw sensed color data to the SAIs, resulting in sparsely distributed color pixels on 2D grids, and perform demosaicking at the receiver after decoding. To exploit spatial correlation among the sparse pixels, we propose a novel intra-prediction scheme, where the prediction kernel is determined according to the local gradient estimated from already coded neighboring pixel blocks. We then connect the pixels by forming a graph, modeling the prediction residuals statistically as a Gaussian Markov Random Field (GMRF). The optimal edge weights are computed via a graph learning method using a set of training SAIs. The residual data is encoded via low-complexity GLT. Experiments show that at high PSNRsimportant for archiving and instant storage scenarios-our method outperformed significantly a conventional light field image coding scheme with demosaicking followed by High Efficiency Video Coding (HEVC).

Geometry-Aware View ReconstructionNetwork for Light Field ImageCompression

“…Santos et al [33] studied three 4D prediction and partition modes for LF lossless encoding including an hexadeca tree, a 4D partition using a quadtree, and a conventional 2D partition with multiple references. Chao et al [34] described a LF coding method based on graph based lifting transform, where redundancies and distortions were suppressed during raw data compression process. Ahmad et al [24] divided the SAIs into two classes including key views and decimated views.…”

Supporting Wider Baseline Light Fields in JPEG Pleno With a Novel Slanted 4D-DCT Coding Mode

“…In [33], selected reference views are coded with HEVC while the remaining views are predicted using a linear combination of the decoded reference views and their residuals encoded using Principal Component Analysis (PCA). Along a different line, the light field coding method in [34] adopts a graphbased lifting transform, thus introducing an intra-prediction approach that operates on the non-demosaicked lenslet PIs.…”

Hybrid Model-Based / Data-Driven Graph Transform for Image Coding

“…We focus instead on the problem of insufficient statistics when coding non-stationary images, and propose a hybrid model-based / datadriven transform towards a good tradeoff in data adaptivity and stability. Compression using graph transforms [13,14,15,16,17,18]-transformation of signals on graphs from the nodal domain to the graph spectral domain-have been studied in the graph signal processing (GSP) literature [19,20] in the past decade. Our graph transform is unique in that our GLASSO-based optimization allows the optimal combination of K model-based eigenvectors with N −K remaining ones computed from data, resulting in a stable transform.…”