LDMIC: Learning-based Distributed Multi-view Image Coding

Abstract

Multi-view image compression plays a critical role in 3D-related applications. Existing methods adopt a predictive coding architecture, which requires joint encoding to compress the corresponding disparity as well as residual information. This demands collaboration among cameras and enforces the epipolar geometric constraint between different views, which makes it challenging to deploy these methods in distributed camera systems with randomly overlapping fields of view. Meanwhile, distributed source coding theory indicates that efficient data compression of correlated sources can be achieved by independent encoding and joint decoding, which motivates us to design a learning-based distributed multi-view image coding (LDMIC) framework. With independent encoders, LDMIC introduces a simple yet effective joint context transfer module based on the cross-attention mechanism at the decoder to effectively capture the global inter-view correlations, which is insensitive to the geometric relationships between images. Experimental results show that LDMIC significantly outperforms both traditional and learningbased MIC methods while enjoying fast encoding speed. Code will be released at https://github.com/Xinjie-Q/LDMIC.

SASIC: Stereo Image Compression with Latent Shifts and Stereo Attention

“…3.4) on any dataset of stereo image pairs. Unlike other recent methods [11,45], the proposed method does not include any autoregressive components, which allows for fast encoding and decoding (see Sec. 4.4).…”

“…The images have a resolution of 2048×1024 and are divided into 2975 training, 500 validation, and 1525 test image pairs. Following conventions [45,42] we crop 64, 256 and 128 pixels from the top, bottom and sides respectively to remove car parts and artefacts from the rectification process. The InStereo2k dataset contains 2060 stereo images of indoor scenes.…”

“…We also report scores for the learned stereo compression methods DSIC [24], HESIC [11], and SASIC [42] from the respective papers. For LDMIC [45] we show the reported scores for the full LDMIC method, which includes an autoregressive context model, and a smaller version LDMIC (fast) without the autoregressive components.…”

“…This is further enhanced by using stereo attention between the two images in the common decoder. Cross-attention in the decoder is also used in the recently proposed distributed multiview method LDMIC by Zhang et al [45]. Contrary to our method, they employ global encoder-to-decoder crossattention and a single image autoregressive entropy model.…”

“…7 depicts the average encoding and decoding times of our method against other methods on the InStereo2k dataset. The conventional methods BPG, HEVC and MV-HEVC were evaluated on an Intel Xeon Gold 6230R pro-cessor with a single core (times are taken from Zhang et al times [45]). For LDMIC we show their reported encoding and decoding times [45] (measured on an NVIDIA RTX 3090 GPU).…”