Learning-Based Practical Light Field Image Compression Using A Disparity-Aware Model

Abstract

Light field technology has increasingly attracted the attention of the research community with its many possible applications. The lenslet array in commercial plenoptic cameras helps capture both the spatial and angular information of light rays in a single exposure. While the resulting high dimensionality of light field data enables its superior capabilities, it also impedes its extensive adoption. Hence, there is a compelling need for efficient compression of light field images. Existing solutions are commonly composed of several separate modules, some of which may not have been designed for the specific structure and quality of light field data. This increases the complexity of the codec and results in impractical decoding runtimes. We propose a new learning-based, disparity-aided model for compression of 4D light field images capable of parallel decoding. The model is endto-end trainable, eliminating the need for hand-tuning separate modules and allowing joint learning of rate and distortion. The disparity-aided approach ensures the structural integrity of the reconstructed light fields. Comparisons with the state of the art show encouraging performance in terms of PSNR and MS-SSIM metrics. Also, there is a notable gain in the encoding and decoding runtimes. Source code is available at https://moha23.github.io/LF-DAAE.

View Sub-sampling and Reconstruction for Efficient Light Field Compression

“…One common approach is utilizing light field reconstruction methods to complete view-subsampled light fields, i.e. compressing only a subset of the original views , Zhao and Chen, 2017, Viola et al, 2018, Jiang et al, 2017b, Jiang et al, 2017a. Various methods use a video codec such as HEVC as coding component, while JPEG launched the JPEG Pleno initiative for standardizing compression of plenoptic data including light fields [Ebrahimi et al, 2016].…”

“…Singh et. al. introduced an end-to-end disparity-aware 3D-CNN for light field compression, which utilizes the disparity information between views and the middle view [Singh and Rameshan, 2021]. Other works target light field compression using adversarial learning , Bakir et al, 2020, Liu et al, 2021.…”