Light Field Image Coding Using VVC Standard and View Synthesis Based on Dual Discriminator GAN

Bakir, Nader; Hamidouche, Wassim; Fezza, Sid Ahmed; Samrouth, Khouloud; Déforges, Olivier

doi:10.1109/tmm.2021.3068563

Cited by 17 publications

(7 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bakir et al 29 designed a dual discriminator GAN to synthesize the dropped SAIs at the decoder side. Subsequently, this work was improved in 30 by adding a multi-view quality enhancement network to ensure the reconstruction qualities of synthesized SAIs. Liu et al 31 constructed a multi-disparity geometry structure of sparse SAIs at the decoder side and then put forward a multi-stream view reconstruction network to reconstruct the entire LF.…”

Section: Related Workmentioning

confidence: 99%

“…While the SAI array based methods [18][19][20][21][22][23][24] intent to enhance the compression performance by eliminating redundancies of adjacent SAIs. Wherein, based on the wide applications of Convolutional Neural Network (CNN) in LF image processing, learning based LF view reconstruction methods [25][26][27][28][29][30][31] are introduced into the LF compression. The main idea of learning based LF compression method is to encode sparsely-sampled LF SAIs at the encoder side and synthesize the rest of SAIs with learning based view reconstruction at the decoder side.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Geometry-aware view reconstruction network for light field image compression

Zhang

Wan

Mao

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Light Field (LF) imaging empowers many attractive applications by simultaneously recording spatial and angular information of light rays. In order to meet the challenges of LF storage and transmission, many view reconstruction-based LF compression methods are put forward. However, occlusion issue and under-exploitation of LF rich structure information limit the view reconstruction qualities, which further influence LF compression efficiency. In order to alleviate these problems, in this paper, we propose a geometry-aware view reconstruction network for LF compression. In our method, only sparsely-sampled LF views are encoded, which are further used as priors to reconstruct the un-sampled LF views at the decoder side. The proposed reconstruction process contains two stages including geometry-aware reconstruction and texture refinement. The geometry-aware reconstruction stage utilizes a multi-stream framework, which can fully explore LF spatial-angular, location and geometry information. The texture refinement stage can adequately fuse such rich LF information to further improve LF reconstruction quality. Comprehensive experimental results validate the superiority of the proposed method. The rate-distortion performance and the perceptual quality of reconstructed views further demonstrate that the proposed method can save more bitrate while increasing LF reconstruction quality.

show abstract

Section: Related Workmentioning

confidence: 99%

mentioning

confidence: 99%

Geometry-aware view reconstruction network for light field image compression

Zhang

Wan

Mao

et al. 2022

Sci Rep

View full text Add to dashboard Cite

show abstract

“…With block motion estimation, discrete cosine transform (DCT), entropy coding, and other related technologies, these conventional video compression methods have achieved stable performance in exploiting data redundancy. Based on the general framework of these traditional methods, several deep neural networks (DNN) have been designed into the hybrid framework as sub-modules [3], [4]. These kinds of solutions aim to improve the performance of certain particular modules of the whole compression framework.…”

Section: Introductionmentioning

confidence: 99%

Learned Video Compression via Heterogeneous Deformable Compensation Network

Wang¹,

Chen²,

Chen³

2022

Preprint

View full text Add to dashboard Cite

Learned video compression has recently emerged as an essential research topic in developing advanced video compression technologies, where motion compensation is considered one of the most challenging issues. In this paper, we propose a learned video compression framework via heterogeneous deformable compensation strategy (HDCVC) to tackle the problems of unstable compression performance caused by singlesize deformable kernels in downsampled feature domain. More specifically, instead of utilizing optical flow warping or single-sizekernel deformable alignment, the proposed algorithm extracts features from the two adjacent frames to estimate contentadaptive heterogeneous deformable (HetDeform) kernel offsets. Then we transform the reference features with the HetDeform convolution to accomplish motion compensation. Moreover, we design a Spatial-Neighborhood-Conditioned Divisive Normalization (SNCDN) to reduce spatial statistic dependencies and achieve more effective data Gaussianization combined with the Generalized Divisive Normalization. Furthermore, we propose a multi-frame enhanced reconstruction module for exploiting context and temporal information for final quality enhancement. Experimental results indicate that HDCVC achieves superior performance than the recent state-of-the-art learned video compression approaches.

show abstract

“…The discarded views are synthesized using the GNN. Bakir et al [24] use VVC's temporal scalability structure to encode key views which are then fed to a GAN to synthesize the remaining views.…”

Section: Introductionmentioning

confidence: 99%

Advanced Scalability for Light Field Image Coding

Amirpour

Guillemot

Ghanbari

et al. 2022

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Light field imaging, which captures both spatial and angular information, improves user immersion by enabling post-capture actions, such as refocusing and changing view perspective. However, light fields represent very large volumes of data with a lot of redundancy that coding methods try to remove. State-of-the-art coding methods indeed usually focus on improving compression efficiency and overlook other important features in light field compression such as scalability. In this paper, we propose a novel light field image compression method that enables (i) viewport scalability, (ii) quality scalability, (iii) spatial scalability, (iv) random access, and (v) uniform quality distribution among viewports, while keeping compression efficiency high. To this end, light fields in each spatial resolution are divided into sequential viewport layers, and viewports in each layer are encoded using the previously encoded viewports. In each viewport layer, the available viewports are used to synthesize intermediate viewports using a video interpolation deep learning network. The synthesized views are used as virtual reference images to enhance the quality of intermediate views. An image super-resolution method is applied to improve the quality of the lower spatial resolution layer. The super-resolved images are also used as virtual reference images to improve the quality of the higher spatial resolution layer. The proposed structure also improves the flexibility of light field streaming, provides random access to the viewports, and increases error resiliency. The experimental results demonstrate that the proposed method achieves a high compression efficiency and it can adapt to the display type, transmission channel, network condition, processing power, and user needs.

show abstract

Light Field Image Coding Using VVC Standard and View Synthesis Based on Dual Discriminator GAN

Cited by 17 publications

References 46 publications

Geometry-aware view reconstruction network for light field image compression

Geometry-aware view reconstruction network for light field image compression

Learned Video Compression via Heterogeneous Deformable Compensation Network

Advanced Scalability for Light Field Image Coding

Contact Info

Product

Resources

About