Interpreting plenoptic images as multi-view sequences for improved compression

Abstract

Over the last decade, advancements in optical devices have made it possible for new novel image acquisition technologies to appear. Angular information for each spatial point is acquired in addition to the spatial information of the scene that enables 3D scene reconstruction and various post-processing effects. Current generation of plenoptic cameras spatially multiplex the angular information, which implies an increase in image resolution to retain the level of spatial information gathered by conventional cameras. In this work, the resulting plenoptic image is interpreted as a multi-view sequence that is efficiently compressed using the multi-view extension of high efficiency video coding (MV-HEVC). A novel twodimensional weighted prediction and rate allocation scheme is proposed to adopt the HEVC compression structure to the plenoptic image properties. The proposed coding approach is a response to ICIP 2017, Grand Challenge on Light Field Image Coding and the compression results are put in contrast to the state-of-art in plenoptic image compression presented at of the ICME 2016 Grand Challenge on Light-field Image Compression. The proposed scheme outperforms all contestants in the ICME Grand Challenge with a significant improvements in compression efficiency, i.e. with an average PSNR gain of 7.7 dB over reference JPEG image compression.

VALID: Visual quality Assessment for Light field Images Dataset

“…Five LF lenslet images were chosen from a publicly available LF image dataset, namely I01 = Bikes, I02 = Danger de Mort, I04 = Stone Pillars Outside, I09 = Fountain & Vincent 2 and I10 = Friends 1 [3]. The images were carefully selected from those commonly used in literature [7], [9], [10], to provide a variety of scenarios, containing a wide range of details that would be challenging for the compression algorithms in terms of texture and disparity encoding. From each lenslet image, 15 × 15 perspective views of 625 × 434 pixels and depth of 10 bits per color channel were obtained, using the Light Field toolbox v0.4 [11], [12].…”

“…In [8] authors encode a subset of the perspective views using HEVC, adopting a linear approximation prior to estimate the non-encoded views. In [9] authors arrange the perspective views into a multiview structure that can be exploited by the corresponding extension of HEVC, namely MV-HEVC. They also propose a rate allocation scheme to progressively assign the QPs in order to optimize the performance.…”

Compression scheme for sparsely sampled light field data based on pseudo multi-view sequences

“…The quantization offset of a frame is estimated based on its distance and view-wise decoding order relative to the reference frame. The methodology was previously proposed for plenoptic image compression 14 and in this work it is extended for LF captured with multi-camera system . The equation (1) presents the quantization offset estimation process for each frame.…”

Quality Assessment Of Compression Solutions for Icip 2017 Grand Challenge on Light Field Image Coding

“…They use HEVC to encode and transmit part of the views, while non-encoded views are estimated by solving an optimization problem. For algorithm P 02, authors arrange the perspective views into a multiview structure that can be exploited by the corresponding extension of HEVC, namely MV-HEVC [5]. They also propose a rate allocation scheme to progressively assign the Quantization Parameters (QP) in order to optimize the performance.…”