Deep Stereo Image Compression via Bi-directional Coding

Lei, Jianjun; Liu, Xiangrui; Peng, Bo; Jin, Dengchao; Li, Wanqing; Gu, Jingxiao

doi:10.1109/cvpr52688.2022.01905

Cited by 12 publications

(5 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this case, the PSNR with (without) the JCT module at the encoder (decoder) improves (drops) by about 0.16dB (0.73dB) at the same bpp level. We further report the compression results when the JCT module is directly replaced by other inter-view fusion operations such as concatenation in Mital et al (2022b), stereo attention module (SAM) in Wödlinger et al (2022) and bi-directional contextual transform module (Bi-CTM) in Lei et al (2022). These operations lead to an increase of the bitrate by 32.73%, 27.99%, 10.11% compared with our method.…”

Section: Ablation Studymentioning

confidence: 93%

“…We also test MV-HEVC (Tech et al, 2015) with the multi-view intra mode. Apart from that, we report the results of several recent DNN-based stereo image codecs on the InStereo2K and Cityscapes datasets, including DSIC (Liu et al, 2019), two variants of HESIC (Deng et al, 2021), BCSIC (Lei et al, 2022), and SASIC (Wödlinger et al, 2022).…”

Section: Methodsmentioning

confidence: 99%

“…Since these methods are still in the development stage and only support YUV420 format, they are uncompetitive against single image codecs that allow the YUV444 or RGB format. Meanwhile, existing learningbased MIC approaches (Liu et al, 2019;Deng et al, 2021;Wödlinger et al, 2022;Lei et al, 2022) mainly focus on stereo images, and it is difficult to effectively extend them to the general multi-view scenario. Moreover, they can only handle a fixed number of views.…”

Section: Related Workmentioning

confidence: 99%

“…In particular, Liu et al (2019), Deng et al (2021) and Wödlinger et al (2022), mimicking traditional MIC techniques, adopt a unidirectional coding mechanism and explicitly utilize the disparity compensation prediction in the pixel/feature space to reduce the inter-view redundancy. Meanwhile, Lei et al (2022) introduces a bi-directional coding framework, called as BCSIC, to jointly compress left and right images simultaneously for exploring the content dependency between the stereo pair. These rudimentary studies demonstrate the potentials of deep neural networks (DNNs) in saving significant bit-rate for MIC.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

LDMIC: Learning-based Distributed Multi-view Image Coding

Zhang¹,

Shao²,

Zhang³

2023

Preprint

View full text Add to dashboard Cite

Multi-view image compression plays a critical role in 3D-related applications. Existing methods adopt a predictive coding architecture, which requires joint encoding to compress the corresponding disparity as well as residual information. This demands collaboration among cameras and enforces the epipolar geometric constraint between different views, which makes it challenging to deploy these methods in distributed camera systems with randomly overlapping fields of view. Meanwhile, distributed source coding theory indicates that efficient data compression of correlated sources can be achieved by independent encoding and joint decoding, which motivates us to design a learning-based distributed multi-view image coding (LDMIC) framework. With independent encoders, LDMIC introduces a simple yet effective joint context transfer module based on the cross-attention mechanism at the decoder to effectively capture the global inter-view correlations, which is insensitive to the geometric relationships between images. Experimental results show that LDMIC significantly outperforms both traditional and learningbased MIC methods while enjoying fast encoding speed. Code will be released at https://github.com/Xinjie-Q/LDMIC.

show abstract

Section: Ablation Studymentioning

confidence: 93%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

LDMIC: Learning-based Distributed Multi-view Image Coding

Zhang¹,

Shao²,

Zhang³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Invertible neural networkbased architecture (Cai et al 2022;Helminger et al 2021;Ho et al 2021;Ma et al 2019Ma et al , 2022aXie, Cheng, and Chen 2021) and transformer-based architecture (Qian et al 2022;Zhu, Yang, and Cohen 2022;Zou, Song, and Zhang 2022;Liu, Sun, and Katto 2023) also have been utilized to enhance the modeling capacity of the transforms. Some other works aim to improve the efficiency of entropy coding, e.g., scale hyperprior entropy model (Ballé et al 2018), channel-wise entropy model (Minnen and Singh 2020), context model (Lee, Cho, and Beack 2019;Mentzer et al 2018;Minnen, Ballé, and Toderici 2018), 3D-context model (Guo et al 2020b), multi-scale hyperprior entropy model (Hu et al 2022), discretized Gaussian mixture model (Cheng et al 2020), checkerboard context model (He et al 2021), split hierarchical variational compression (SHVC) (Ryder et al 2022), information transformer (Informer) entropy model (Kim, Heo, and Lee 2022), bi-directional conditional entropy model (Lei et al 2022), unevenly grouped space-channel context model (ELIC) (He et al 2022), neural data-dependent transform (Wang et al 2022a), multi-level cross-channel entropy model (Guo et al 2022), and multivariate Gaussian mixture model . By constructing more accurate entropy models, these methods have achieved greater compression efficiency.…”

Section: Related Workmentioning

confidence: 99%

Make Lossy Compression Meaningful for Low-Light Images

Cai,

Chen,

Zhong

et al. 2024

AAAI

View full text Add to dashboard Cite

Low-light images frequently occur due to unavoidable environmental influences or technical limitations, such as insufficient lighting or limited exposure time. To achieve better visibility for visual perception, low-light image enhancement is usually adopted. Besides, lossy image compression is vital for meeting the requirements of storage and transmission in computer vision applications. To touch the above two practical demands, current solutions can be categorized into two sequential manners: ``Compress before Enhance (CbE)'' or ``Enhance before Compress (EbC)''. However, both of them are not suitable since: (1) Error accumulation in the individual models plagues sequential solutions. Especially, once low-light images are compressed by existing general lossy image compression approaches, useful information (e.g., texture details) would be lost resulting in a dramatic performance decrease in low-light image enhancement. (2) Due to the intermediate process, the sequential solution introduces an additional burden resulting in low efficiency. We propose a novel joint solution to simultaneously achieve a high compression rate and good enhancement performance for low-light images with much lower computational cost and fewer model parameters. We design an end-to-end trainable architecture, which includes the main enhancement branch and the signal-to-noise ratio (SNR) aware branch. Experimental results show that our proposed joint solution achieves a significant improvement over different combinations of existing state-of-the-art sequential ``Compress before Enhance'' or ``Enhance before Compress'' solutions for low-light images, which would make lossy low-light image compression more meaningful. The project is publicly available at: https://github.com/CaiShilv/Joint-IC-LL.

show abstract