SASIC: Stereo Image Compression with Latent Shifts and Stereo Attention

Abstract

In this paper, we present ECSIC, a novel learned method for stereo image compression. Our proposed method compresses the left and right images in a joint manner by exploiting the mutual information between the images of the stereo image pair using a novel stereo cross attention (SCA) module and two stereo context modules. The SCA module performs cross-attention restricted to the corresponding epipolar lines of the two images and processes them in parallel. The stereo context modules improve the entropy estimation of the second encoded image by using the first image as a context. We conduct an extensive ablation study demonstrating the effectiveness of the proposed modules and a comprehensive quantitative and qualitative comparison with existing methods. ECSIC achieves state-of-theart performance among stereo image compression models on the two popular stereo image datasets Cityscapes and InStereo2k while allowing for fast encoding and decoding, making it highly practical for real-time applications.

LDMIC: Learning-based Distributed Multi-view Image Coding

“…In this case, the PSNR with (without) the JCT module at the encoder (decoder) improves (drops) by about 0.16dB (0.73dB) at the same bpp level. We further report the compression results when the JCT module is directly replaced by other inter-view fusion operations such as concatenation in Mital et al (2022b), stereo attention module (SAM) in Wödlinger et al (2022) and bi-directional contextual transform module (Bi-CTM) in Lei et al (2022). These operations lead to an increase of the bitrate by 32.73%, 27.99%, 10.11% compared with our method.…”

“…(2) Joint model has access to a set of multi-view images and explicitly utilizes the inter-view redundancy to achieve a high compression ratio. According to performance comparisons in Wödlinger et al (2022), conventional video standards can be applied in the MIC, where each set of multi-view images is compressed as a multi-frame video sequence by using both HEVC (Sullivan et al, 2012) and VVC (Bross et al, 2021) with lowdelay P configuration as well as YUV444 input format. We also test MV-HEVC (Tech et al, 2015) with the multi-view intra mode.…”

“…We also test MV-HEVC (Tech et al, 2015) with the multi-view intra mode. Apart from that, we report the results of several recent DNN-based stereo image codecs on the InStereo2K and Cityscapes datasets, including DSIC (Liu et al, 2019), two variants of HESIC (Deng et al, 2021), BCSIC (Lei et al, 2022), and SASIC (Wödlinger et al, 2022).…”

“…Since these methods are still in the development stage and only support YUV420 format, they are uncompetitive against single image codecs that allow the YUV444 or RGB format. Meanwhile, existing learningbased MIC approaches (Liu et al, 2019;Deng et al, 2021;Wödlinger et al, 2022;Lei et al, 2022) mainly focus on stereo images, and it is difficult to effectively extend them to the general multi-view scenario. Moreover, they can only handle a fixed number of views.…”

“…Inspired by the great success of learning-based single image compression (Ballé et al, 2017;Cheng et al, 2020), several recent works have investigated the application of deep learning techniques to stereo image coding, a special case of MIC. In particular, Liu et al (2019), Deng et al (2021) and Wödlinger et al (2022), mimicking traditional MIC techniques, adopt a unidirectional coding mechanism and explicitly utilize the disparity compensation prediction in the pixel/feature space to reduce the inter-view redundancy. Meanwhile, Lei et al (2022) introduces a bi-directional coding framework, called as BCSIC, to jointly compress left and right images simultaneously for exploring the content dependency between the stereo pair.…”