Learning Accurate Entropy Model with Global Reference for Image Compression

Qian, Yichen; Tan, Zhiyu; Sun, Xinli; Lin, Minren; Li, Dongyang; Sun, Zhi-Hong; Li, Hao; Jin, Rong

doi:10.48550/arxiv.2010.08321

Cited by 3 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, the auto-regressive prior [40] proposes using neighbour contexts to capture spatial correlation. Recently works [18,26,44,45] propose extracting global or long-range contexts to further boost performance. These show more diverse contexts bring substantial coding gain for neural image codec.…”

Section: Neural Image Compressionmentioning

confidence: 99%

Neural Video Compression with Diverse Contexts

Li¹,

Li²,

Lü³

2023

Preprint

View full text Add to dashboard Cite

For any video codecs, the coding efficiency highly relies on whether the current signal to be encoded can find the relevant contexts from the previous reconstructed signals. Traditional codec has verified more contexts bring substantial coding gain, but in a time-consuming manner. However, for the emerging neural video codec (NVC), its contexts are still limited, leading to low compression ratio. To boost NVC, this paper proposes increasing the context diversity in both temporal and spatial dimensions. First, we guide the model to learn hierarchical quality patterns across frames, which enriches long-term and yet highquality temporal contexts. Furthermore, to tap the potential of optical flow-based coding framework, we introduce a group-based offset diversity where the cross-group interaction is proposed for better context mining. In addition, this paper also adopts a quadtree-based partition to increase spatial context diversity when encoding the latent representation in parallel. Experiments show that our codec obtains 23.5% bitrate saving over previous SOTA NVC. Better yet, our codec has surpassed the under-developing next generation traditional codec/ECM in both RGB and YUV420 colorspaces, in terms of PSNR. The codes are at https://github.com/microsoft/DCVC.

show abstract

Section: Neural Image Compressionmentioning

confidence: 99%

Neural Video Compression with Diverse Contexts

Li¹,

Li²,

Lü³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…More works [5,6,34,22,15,25] pay attention to the entropy network and obtain obvious improvements. Ballé et al [5] use a fully factorized prior to minimize the entropy of the elements of the whole latent representation.…”

Section: Learned Image Compressionmentioning

confidence: 99%

“…This kind of method outperforms the top standard image codec (BPG) on both the PSNR and MS-SSIM distortion metrics. After that, Qian [25] builds up a global relevance throughout latent features to further explore the relationship in the whole picture. All these works show that designing an accurate entropy model is crucial or even the only thing for learned image compression.…”

Section: Learned Image Compressionmentioning

confidence: 99%

Spatiotemporal Entropy Model is All You Need for Learned Video Compression

Sun¹,

Tan²,

Sun³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The framework of dominant learned video compression methods is usually composed of motion prediction modules as well as motion vector and residual image compression modules, suffering from its complex structure and error propagation problem. Approaches have been proposed to reduce the complexity by replacing motion prediction modules with implicit flow networks. Error propagation aware training strategy is also proposed to alleviate incremental reconstruction errors from previously decoded frames. Although these methods have brought some improvement, little attention has been paid to the framework itself. Inspired by the success of learned image compression through simplifying the framework with a single deep neural network, it is natural to expect a better performance in video compression via a simple yet appropriate framework. Therefore, we propose a framework to directly compress raw-pixel frames (rather than residual images), where no extra motion prediction module is required. Instead, an entropy model is used to estimate the spatiotemporal redundancy in a latent space rather than pixel level, which significantly reduces the complexity of the framework. Specifically, the whole framework is a compression module, consisting of a unified auto-encoder which produces identically distributed latents for all frames, and a spatiotemporal entropy estimation model to minimize the entropy of these latents. Experiments showed that the proposed method outperforms state-of-the-art (SOTA) performance under the metric of multiscale structural similarity (MS-SSIM) and achieves competitive results under the metric of PSNR.

show abstract

Learned Block-based Hybrid Image Compression

Wu¹,

Li²,

Zhang³

et al. 2020

Preprint

View full text Add to dashboard Cite

Learned image compression based on neural networks have made huge progress thanks to its superiority in learning better representation through non-linear transformation. Different from traditional hybrid coding frameworks, that are commonly block-based, existing learned image codecs usually process the images in a full-resolution manner thus not supporting acceleration via parallelism and explicit prediction. Compared to learned image codecs, traditional hybrid coding frameworks are in general hand-crafted and lack the adaptability of being optimized according to heterogeneous metrics. Therefore, in order to collect their good qualities and offset their weakness, we explore a learned block-based hybrid image compression (LBHIC) framework, which achieves a win-win between coding performance and efficiency. Specifically, we introduce block partition and explicit learned predictive coding into learned image compression framework. Compared to prediction through linear weighting of neighbor pixels in traditional codecs, our contextual prediction module (CPM) is designed to better capture long-range correlations by utilizing the strip pooling to extract the most relevant information in neighboring latent space. Moreover, to alleviate blocking artifacts, we further propose a boundary-aware postprocessing module (BPM) with the importance of edge taken into account. Extensive experiments demonstrate that the proposed LBHIC codec outperforms state-of-the-art image compression methods in terms of both PSNR and MS-SSIM metrics and promises obvious time-saving.

show abstract

Learning Accurate Entropy Model with Global Reference for Image Compression

Cited by 3 publications

References 14 publications

Neural Video Compression with Diverse Contexts

Neural Video Compression with Diverse Contexts

Spatiotemporal Entropy Model is All You Need for Learned Video Compression

Learned Block-based Hybrid Image Compression

Contact Info

Product

Resources

About