Video quality assessment accounting for temporal visual masking of local flicker

Choi, Lark Kwon; Bovik, Alan C.

doi:10.1016/j.image.2018.06.009

Cited by 29 publications

(11 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper addresses this gap by building on the recent research on HVS-based stereoscopic video quality assessment and generalising it to incorporate for the first time a physiologically inspired model of motion sensitivity. The importance of considering motion sensitivity in enhancing the accuracy of 2D video quality assessment modelling was highlighted in previous research [37], [38]. This importance is particularly magnified for realising a precise 3D video quality assessment model.…”

Section: Stereoscopic Video Quality Metricsmentioning

confidence: 95%

Full-Reference Stereoscopic Video Quality Assessment Using a Motion Sensitive HVS Model

Galkandage

Calic

Dogan

et al. 2021

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Stereoscopic video quality assessment has become a major research topic in recent years. Existing stereoscopic video quality metrics are predominantly based on stereoscopic image quality metrics extended to the time domain via for example temporal pooling. These approaches do not explicitly consider the motion sensitivity of the Human Visual System (HVS). To address this limitation, this paper introduces a novel HVS model inspired by physiological findings characterising the motion sensitive response of complex cells in the primary visual cortex (V1 area). The proposed HVS model generalises previous HVS models, which characterised the behaviour of simple and complex cells but ignored motion sensitivity, by estimating optical flow to measure scene velocity at different scales and orientations. The local motion characteristics (direction and amplitude) are used to modulate the output of complex cells. The model is applied to develop a new type of full-reference stereoscopic video quality metrics which uniquely combine non-motion sensitive and motion sensitive energy terms to mimic the response of the HVS. A tailored two-stage multi-variate stepwise regression algorithm is introduced to determine the optimal contribution of each energy term. The two proposed stereoscopic video quality metrics are evaluated on three stereoscopic video datasets. Results indicate that they achieve average correlations with subjective scores of 0.9257 (PLCC), 0.9338 and 0.9120 (SRCC), 0.8622 and 0.8306 (KRCC), and outperform previous stereoscopic video quality metrics including other recent HVS-based metrics.

show abstract

Section: Stereoscopic Video Quality Metricsmentioning

confidence: 95%

Full-Reference Stereoscopic Video Quality Assessment Using a Motion Sensitive HVS Model

Galkandage

Calic

Dogan

et al. 2021

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

“…Temporal-memory effect is another important clue for designing objective VQA models (Park et al 2013;Seshadrinathan and Bovik 2011;Xu et al 2014;Choi and Bovik 2018;Kim et al 2018). It induces that video quality rating is influenced by historic memory.…”

Section: Modeling Of Temporal-memory Effectmentioning

confidence: 99%

Unified Quality Assessment of in-the-Wild Videos with Mixed Datasets Training

Jiang

2021

Int J Comput Vis

115

View full text Add to dashboard Cite

Video quality assessment (VQA) is an important problem in computer vision. The videos in computer vision applications are usually captured in the wild. We focus on automatically assessing the quality of in-the-wild videos, which is a challenging problem due to the absence of reference videos, the complexity of distortions, and the diversity of video contents. Moreover, the video contents and distortions among existing datasets are quite different, which leads to poor performance of datadriven methods in the cross-dataset evaluation setting. To improve the performance of quality assessment models, we borrow intuitions from human perception, specifically, content dependency and temporal-memory effects of human visual system. To face the cross-dataset evaluation challenge, we explore a mixed datasets training strategy for training a single VQA model with multiple datasets. The proposed unified framework explicitly includes three stages: relative quality assessor, nonlinear mapping, and dataset-specific perceptual scale alignment, to jointly predict relative quality, perceptual quality, and subjective quality. Experiments are conducted on four publicly available datasets for VQA in the wild, i.e., LIVE-VQC, LIVE-Qualcomm, KoNViD-1k, and CVD2014. The experimental results verify the effectiveness of the mixed datasets training strategy and prove the superior performance of the unified model in comparison with the state-of-the-art models. For reproducible research, we make the PyTorch implementation of our method available at https://github.com/lidq92/MDTVSFA. Keywords Content dependency • In-the-wild videos • Mixed datasets training • Temporal-memory effect • Video quality assessment Communicated by Mei Chen.

show abstract

“…Temporal masking is one of the important aspect of human visual system (HVS), which has proven to an impact on perception of video artifacts. Choi et al, [11], analyzed the influence of motion on the performance of image level quality metrics after dividing LIVE VQA database into two subsets of low-level motions and highlevel motion contents. Their results revealed that many frame-based quality metrics such as PSNR, perform poor in case of high-level motion content.…”

Section: Video Quality Prediction Phasementioning

confidence: 99%

NDNetGaming - development of a no-reference deep CNN for gaming video quality prediction

Utke

Zadtootaghaj

Schmidt

et al. 2020

Multimed Tools Appl

View full text Add to dashboard Cite

Gaming video streaming services are growing rapidly due to new services such as passive video streaming of gaming content, e.g. Twitch.tv, as well as cloud gaming, e.g. Nvidia GeForce NOW and Google Stadia. In contrast to traditional video content, gaming content has special characteristics such as extremely high and special motion patterns, synthetic content and repetitive content, which poses new opportunities for the design of machine learning-based models to outperform the state-of-the-art video and image quality approaches for this special computer generated content. In this paper, we train a Convolutional Neural Network (CNN) based on an objective quality model, VMAF, as ground truth and fine-tuned it based on subjective image quality ratings. In addition, we propose a new temporal pooling method to predict gaming video quality based on frame-level predictions. Finally, the paper also describes how an appropriate CNN architecture can be chosen and how well the model performs on different contents. Our result shows that among four popular network architectures that we investigated, DenseNet performs best for image quality assessment based on the training dataset. By training the last 57 convolutional layers of DenseNet based on VMAF values, we obtained a high performance model to predict VMAF of distorted frames of video games with a Spearman's Rank correlation (SRCC) of 0.945 and Root Mean Score Error (RMSE) of 7.07 on the image level, while achieving a higher performance on the video level leading to a SRCC of 0.967 and RMSE of 5.47 for the KUGVD dataset. Furthermore, we fine-tuned the model based on subjective quality ratings of images from gaming content which resulted in a SRCC of 0.93 and RMSE of 0.46 using one-hold-out cross validation. Finally, on the video level, using the proposed pooling method, the model achieves a very good performance indicated by a SRCC of 0.968 and RMSE of 0.30 for the used gaming video dataset.

show abstract

Video quality assessment accounting for temporal visual masking of local flicker

Cited by 29 publications

References 69 publications

Full-Reference Stereoscopic Video Quality Assessment Using a Motion Sensitive HVS Model

Full-Reference Stereoscopic Video Quality Assessment Using a Motion Sensitive HVS Model

Unified Quality Assessment of in-the-Wild Videos with Mixed Datasets Training

NDNetGaming - development of a no-reference deep CNN for gaming video quality prediction

Contact Info

Product

Resources

About