View Synthesis-based Distributed Light Field Compression

Abstract

Light field imaging is becoming a key technology, which provides users with a realistic visual experience through the capability of dynamic viewpoint shifting. This ability comes at the cost of capturing huge amounts of information, leaving the problem of compression and transmission a challenge. The encoder complexity is the key to achieve efficient coding in conventional light field coding schemes, where a complicated prediction process is essentially used at the encoder side to exploit the redundancy present in the light field image. We employ Distributed Source Coding (DSC) for light field images, which can extensively lift the computational requirement from the encoding side at the expense of increased computational complexity at the decoder side. The efficiency of DSC is heavily dependent on the quality of side information at the decoder. Therefore, we propose to leverage a learningbased view synthesis method, which takes into account the light field structure to generate high-quality side information. We compare our approach to Distributed Video Coding and Distributed Multi-view Video Coding schemes adapted to the light field framework and relevant standard-based approach, and demonstrate that the proposed view synthesis-based approach can achieve similar performance, while substantially reducing the number of key views to be transmitted.

Improved Deep Distributed Light Field Coding

“…In this section, we describe the proposed Distributed Light Field Coding (DLFC) scheme. In our previous work [12], we have utilized the view synthesis approach, proposed by Navarro and Sabater [17], for the prediction of WZ views. Here, we extend our previous work by considering an improvement of the SI generation whose quality directly correlates with the performance of the coding scheme.…”

“…Additionally, a robust fusion method is employed by fusing likelihoods estimated from each SI. The approach can be adapted for light field structures by substituting one angular dimension in place of temporal dimension [12]. PhiCong et al [14] utilize an adaptive strategy to skip WZ decoding process if the synthesized view at the decoder is estimated to have a minimum quality to avoid transmitting bits for that particular view.…”

“…In order to use existing DVC tools, in [14] the LF views are first downsampled to QCIF resolution and then converted to a pseudo video sequence by utilizing a so-called Hybrid scanning order. Mukati et al [12] propose to use a view synthesis-based approach to synthesize light field views at the decoder utilizing only four key views picked from the four corners of the LF in order to reduce the encoding complexity radically. The results show that leveraging high-quality synthesized views provide competitive RD performance compared to the state-of-the-art DMVC approach [13].…”

“…For the sake of completeness, we briefly describe the view synthesis approach used in our previous work [12]. For a more detailed description, the reader may also consult [17].…”

“…In our previous work [12] DLFC-I, we propose replacing a typical optical flow-based prediction scheme with a learning approach to generate high-quality estimates of WZ views while considerably reducing the complexity of the encoder. In this work, we build upon our previous work [12] by further leveraging deep learning approaches for better estimation of SI in the distributed coding scenario. More precisely, we improve the view synthesis performance by considering different arrangements of the reference view and we propose a deep learning-based approach for the estimation of the residual signal.…”