Video Compression Based on Spatio-Temporal Resolution Adaptation

Afonso, Mariana; Zhang, Fan; Bull, David

doi:10.1109/tcsvt.2018.2878952

Cited by 68 publications

(46 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It however remains an underdeveloped research area, with most existing work focused on the improvement of conventional coding tools, such as intra prediction [10,11], inter prediction [12,13] and inloop filters [14,15]. Some work has been reported on the use of deep learning based resolution adaptation methods [6,16,17], which have demonstrated resolution adaptation across a wider bitrate range and offer significant coding gains.…”

Section: Introductionmentioning

confidence: 99%

“…Inspired by our previous work [16], a novel Convolutional Neural Network (CNN) based effective bit depth adaptation approach (EBDA-CNN) is proposed for video compression, which reduces the effective bit depth of an input video before encoding, and reconstructs the original bit depth at the decoder. The data precision and internal bit depth during encoding and decoding remain the same.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Enhanced Video Compression Based on Effective Bit Depth Adaptation

Zhang

Afonso

Bull

2019

2019 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

This paper presents a novel Convolutional Neural Network (CNN) based effective bit depth adaptation approach (EBDA-CNN) for video compression. It applies effective bit depth down-sampling before encoding and reconstructs the original bit depth using a deep CNN based up-sampling method at the decoder. The proposed approach has been integrated with the High Efficiency Video Coding reference software HM 16.20, and evaluated under the Joint Video Exploration Team Common Test Conditions using the Random Access configuration. The results show consistent coding gains on all tested sequences, with an average bitrate saving of 6.4%, based on Bjøntegaard Delta measurements using PSNR.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Enhanced Video Compression Based on Effective Bit Depth Adaptation

Zhang

Afonso

Bull

2019

2019 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…This shift has been largely fuelled by the success of deep CNN architectures for single image super-resolution, that have set the state-of-the-art, with recent architectures like VDSR [15], EDSR [16], FSRCNN [17], DRCN [18] and DBPN [4] achieving several dB higher PSNR in the luminance channel of standard image benchmarks for lossless image upscaling. Thus, Afonso et al propose a spatio-temporal resolution adaptation where a CNN-based super-resolution model is used to reconstruct full-resolution content [19]. Li et al [20] introduce the block adaptive resolution coding framework for intra frame coding, where each block within a frame is either downscaled or coded at original resolution and then upscaled with a trained CNN at the decoder side.…”

Section: Related Workmentioning

confidence: 99%

Deep Video Precoding

Bourtsoulatze¹,

Chadha²,

Fadeev³

et al. 2020

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Several groups worldwide are currently investigating how deep learning may advance the state-of-the-art in image and video coding. An open question is how to make deep neural networks work in conjunction with existing (and upcoming) video codecs, such as MPEG H.264/AVC, H.265/HEVC, VVC, Google VP9 and AOMedia AV1, as well as existing container and transport formats, without imposing any changes at the client side. Such compatibility is a crucial aspect when it comes to practical deployment, especially when considering the fact that the video content industry and hardware manufacturers are expected to remain committed to supporting these standards for the foreseeable future.We propose to use deep neural networks as precoders for current and future video codecs and adaptive video streaming systems. In our current design, the core precoding component comprises a cascaded structure of downscaling neural networks that operates during video encoding, prior to transmission. This is coupled with a precoding mode selection algorithm for each independently-decodable stream segment, which adjusts the downscaling factor according to scene characteristics, the utilized encoder, and the desired bitrate and encoding configuration. Our framework is compatible with all current and future codec and transport standards, as our deep precoding network structure is trained in conjunction with linear upscaling filters (e.g., the bilinear filter), which are supported by all web video players. Results with FHD (1080p) and UHD (2160p) content and widelyused H.264/AVC, H.265/HEVC and VP9 encoders show that coupling such standards with the proposed deep video precoding allows for 15% to 45% rate reduction under encoding configurations and bitrates suitable for video-on-demand adaptive streaming systems. The use of precoding can also lead to encoding complexity reduction, which is essential for cost-effective cloud deployment of complex encoders like H.265/HEVC and VP9, especially when considering the prominence of high-resolution adaptive video streaming.

show abstract

“…Despite the current trend of increasing the capture and display devices spatial resolutions further and further, such resolutions are not necessarily required to represent the critical information of the signal. A well-known strategy consists in downsampling the video prior to encoding and upsampling back to the original resolution after decoding, in an effort to reduce the bitrate and/or the coding complexity for a similar reconstructed output quality [7][8][9][10][11][12][13][14][15]. Several studies have considered applying the down/up-sampling at an image level, showing that coding at a lower resolution performs better, especially at low bitrate [7,8].…”

Section: Introductionmentioning

confidence: 99%

“…Based on the recent advances in the active field of single image Super Resolution (SR) [16,17], several studies have used SR upsampling for ASR-based coding showing increased gains compared to common upsampling filter-banks [11][12][13][14][15]. However, these gains come at the expense of a significant increase in processing complexity, making SR unsuitable for low-complexity encoding/decoding.…”

Section: Introductionmentioning

confidence: 99%

Low-Complexity Scalable Encoder Based on Local Adaptation of the Spatial Resolution

Herrou

Hamidouche

Morin

2019

2019 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

A two-layer low-complexity scalable encoding scheme based on local Adaptive Spatial Resolution (ASR) is proposed. This scheme relies on a block-level spatial resolution adaptation in the enhancement layer encoder. For each block, the optimal resolution is either obtained via a rate-distortion optimization followed by a decision refinement process or by a prediction via motion compensation exploiting the base layer motion vectors. The proposed architecture has been integrated over of the High Efficiency Video Coding (HEVC) reference software (HM16.12) which is used as a base layer encoder. Compared to SHVC, the scalable extension of HEVC, experimental results show bitrate savings of 0.76 % as well as encoding complexity reductions of 47 % for the whole scalable encoder and 96 % for the enhancement layer encoder.

show abstract

Video Compression Based on Spatio-Temporal Resolution Adaptation

Cited by 68 publications

References 20 publications

Enhanced Video Compression Based on Effective Bit Depth Adaptation

Enhanced Video Compression Based on Effective Bit Depth Adaptation

Deep Video Precoding

Low-Complexity Scalable Encoder Based on Local Adaptation of the Spatial Resolution

Contact Info

Product

Resources

About