Learning-Based Just-Noticeable-Quantization- Distortion Modeling for Perceptual Video Coding

Abstract

Conventional predictive video coding-based approaches are reaching the limit of their potential coding efficiency improvements, because of severely increasing computation complexity. As an alternative approach, perceptual video coding (PVC) has attempted to achieve high coding efficiency by eliminating perceptual redundancy, using just-noticeable-distortion (JND) directed PVC. The previous JNDs were modeled by adding white Gaussian noise or specific signal patterns into the original images, which were not appropriate in finding JND thresholds due to distortion with energy reduction. In this paper, we present a novel discrete cosine transform-based energy-reduced JND model, called ERJND, that is more suitable for JND-based PVC schemes. Then, the proposed ERJND model is extended to two learning-based just-noticeable-quantization-distortion (JNQD) models as preprocessing that can be applied for perceptual video coding. The two JNQD models can automatically adjust JND levels based on given quantization step sizes. One of the two JNQD models, called LR-JNQD, is based on linear regression and determines the model parameter for JNQD based on extracted handcraft features. The other JNQD model is based on a convolution neural network (CNN), called CNN-JNQD. To our best knowledge, our paper is the first approach to automatically adjust JND levels according to quantization step sizes for preprocessing the input to video encoders. In experiments, both the LR-JNQD and CNN-JNQD models were applied to high efficiency video coding (HEVC) and yielded maximum (average) bitrate reductions of 38.51% (10.38%) and 67.88% (24.91%), respectively, with little subjective video quality degradation, compared with the input without preprocessing applied.

High Efficiency Video Coding Compliant Perceptual Video Coding Using Entropy Based Visual Saliency Model

“…This shows that the proposed scheme performs well at low data rates. Similarly, the proposed PVC scheme achieves more bit rate reduction as compared to the Sehwan [51] PVC scheme when encoded at QP = 22 and QP = 37, which shows better performance of the proposed scheme at low and high data rates. The proposed PVC scheme DMOS values are close to zero for all QP values when compared with both the PVC schemes.…”

“…The comparison of perceptual video coding schemes available in the literature is a challenging task as each scheme utilizes a different set of video sequences and quality evaluation metrics. For example, Sehwan [51] used six video sequences for evaluation and compared results with HEVC HM 16.17. Similarly, Bae [17] used six video sequences and compared results with HEVC HM 11.0.…”

“…Similarly, Bae [17] used six video sequences and compared results with HEVC HM 11.0. Table 5 presents a comparison in terms of bitrate reduction and DMOS for the video sequences that are common among the proposed, Sehwan [51] and Bae [17] PVC schemes. It is evident that the proposed PVC scheme achieves more bit rate reduction as compared to Bae [17] PVC schemes when encoded at QP = 32 and QP = 37.…”

“…Cont. Performance comparison of the proposed PVC with Sehwan[51] and Bae[17] in terms of bitrate reduction and DMOS values for common videos.…”

A Perceptual Video Coding Based on JND Model

“…When the distortion is less than the human sensitivity threshold, the human eye is imperceptible [3]. In recent years, the JND model has received wide attention in the aspects of video image encoding [4] [5], digital watermarking [6], image quality evaluation [7] and so on. At present, several JND models have been proposed: the JND model based on pixel domain and the JND model based on transform domain.…”