Deep Homography for Efficient Stereo Image Compression

Deng, Xin; Yang, Wenzhe; Yang, Ren; Xu, Mai; Liu, Enpeng; Feng, Qianhan; Timofte, Radu

doi:10.1109/cvpr46437.2021.00154

Cited by 37 publications

(10 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since these methods are still in the development stage and only support YUV420 format, they are uncompetitive against single image codecs that allow the YUV444 or RGB format. Meanwhile, existing learningbased MIC approaches (Liu et al, 2019;Deng et al, 2021;Wödlinger et al, 2022;Lei et al, 2022) mainly focus on stereo images, and it is difficult to effectively extend them to the general multi-view scenario. Moreover, they can only handle a fixed number of views.…”

Section: Related Workmentioning

confidence: 99%

“…Inspired by the great success of learning-based single image compression (Ballé et al, 2017;Cheng et al, 2020), several recent works have investigated the application of deep learning techniques to stereo image coding, a special case of MIC. In particular, Liu et al (2019), Deng et al (2021) and Wödlinger et al (2022), mimicking traditional MIC techniques, adopt a unidirectional coding mechanism and explicitly utilize the disparity compensation prediction in the pixel/feature space to reduce the inter-view redundancy. Meanwhile, Lei et al (2022) introduces a bi-directional coding framework, called as BCSIC, to jointly compress left and right images simultaneously for exploring the content dependency between the stereo pair.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

LDMIC: Learning-based Distributed Multi-view Image Coding

Zhang¹,

Shao²,

Zhang³

2023

Preprint

View full text Add to dashboard Cite

Multi-view image compression plays a critical role in 3D-related applications. Existing methods adopt a predictive coding architecture, which requires joint encoding to compress the corresponding disparity as well as residual information. This demands collaboration among cameras and enforces the epipolar geometric constraint between different views, which makes it challenging to deploy these methods in distributed camera systems with randomly overlapping fields of view. Meanwhile, distributed source coding theory indicates that efficient data compression of correlated sources can be achieved by independent encoding and joint decoding, which motivates us to design a learning-based distributed multi-view image coding (LDMIC) framework. With independent encoders, LDMIC introduces a simple yet effective joint context transfer module based on the cross-attention mechanism at the decoder to effectively capture the global inter-view correlations, which is insensitive to the geometric relationships between images. Experimental results show that LDMIC significantly outperforms both traditional and learningbased MIC methods while enjoying fast encoding speed. Code will be released at https://github.com/Xinjie-Q/LDMIC.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

LDMIC: Learning-based Distributed Multi-view Image Coding

Zhang¹,

Shao²,

Zhang³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Centralized stereo image compression was first considered in the DSIC model in [17], and the HESIC model in [9], in which both the left and right images are available at the encoder, and are jointly compressed. In DSIC, a dense warp field is estimated using disparity estimation between the two images, and warped features from the left image are fed to the encoder and decoder of the right image.…”

Section: Centralized Stereo Compressionmentioning

confidence: 99%

Neural Distributed Image Compression with Cross-Attention Feature Alignment

Mital¹,

Ezgi²,

Ali³

et al. 2022

Preprint

View full text Add to dashboard Cite

We propose a novel deep neural network (DNN) architecture for compressing an image when a correlated image is available as side information only at the decoder side, a special case of the well-known and heavily studied distributed source coding (DSC) problem. In particular, we consider a pair of stereo images, which have overlapping fields of view, captured by a synchronized and calibrated pair of cameras; and therefore, are highly correlated. We assume that one image of the pair is to be compressed and transmitted, while the other image is available only at the decoder. In the proposed architecture, the encoder maps the input image to a latent space using a DNN, quantizes the latent representation, and compresses it losslessly using entropy coding. The proposed decoder extracts useful information common between the images solely from the available side information, as well as a latent representation of the side information. Then, the latent representations of the two images, one received from the encoder, the other extracted locally, along with the locally generated common information, are fed to the respective decoders of the two images. We employ a cross-attention module (CAM) to align the feature maps obtained in the intermediate layers of the respective decoders of the two images, thus allowing better utilization of the side information. We train and demonstrate the effectiveness of the proposed algorithm on various realistic setups, such as KITTI and Cityscape datasets of stereo image pairs. Our results show that the proposed architecture is capable of exploiting the decoder-only side information in a more efficient manner as it outperforms previous works. We also show that the proposed method is able to provide significant gains even in the case of uncalibrated and unsynchronized camera array use cases.

show abstract

“…The result is a

matrix representing a set of equations that satisfy the transformation function from the first image plane to the second image plane, as discussed above. For a detailed discussion on homography matrix, see previous studies [ 12 , 13 , 14 ].

…”

Section: Introductionmentioning

confidence: 99%

Automatic Sequential Stitching of High-Resolution Panorama for Android Devices Using Precapture Feature Detection and the Orientation Sensor

Yaseen

Kwon²,

Lee³

et al. 2023

Sensors

View full text Add to dashboard Cite

Image processing on smartphones, which are resource-limited devices, is challenging. Panorama generation on modern mobile phones is a requirement of most mobile phone users. This paper presents an automatic sequential image stitching algorithm with high-resolution panorama generation and addresses the issue of stitching failure on smartphone devices. A robust method is used to automatically control the events involved in panorama generation from image capture to image stitching on Android operating systems. The image frames are taken in a firm spatial interval using the orientation sensor included in smartphone devices. The features-based stitching algorithm is used for panorama generation, with a novel modification to address the issue of stitching failure (inability to find local features causes this issue) when performing sequential stitching over mobile devices. We also address the issue of distortion in sequential stitching. Ultimately, in this study, we built an Android application that can construct a high-resolution panorama sequentially with automatic frame capture based on an orientation sensor and device rotation. We present a novel research methodology (called “Sense-Panorama”) for panorama construction along with a development guide for smartphone developers. Based on our experiments, performed by Samsung Galaxy SM-N960N, which carries system on chip (SoC) as Qualcomm Snapdragon 845 and a CPU of 4 × 2.8 GHz Kyro 385, our method can generate a high-resolution panorama. Compared to the existing methods, the results show improvement in visual quality for both subjective and objective evaluation.

show abstract

Deep Homography for Efficient Stereo Image Compression

Cited by 37 publications

References 26 publications

LDMIC: Learning-based Distributed Multi-view Image Coding

LDMIC: Learning-based Distributed Multi-view Image Coding

Neural Distributed Image Compression with Cross-Attention Feature Alignment

Automatic Sequential Stitching of High-Resolution Panorama for Android Devices Using Precapture Feature Detection and the Orientation Sensor

Contact Info

Product

Resources

About