FAVER: Blind Quality Prediction of Variable Frame Rate Videos

Zheng, Qi; Tu, Zhengzhong; Madhusudana, Pavan C.; Zeng, Xiaoyang; Bovik, Alan C.; Fan, Yibo

doi:10.48550/arxiv.2201.01492

Cited by 4 publications

(11 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous studies evaluate the video quality either using the original spatial resolution or a fixed resized spatial resolution, which ignore that videos are naturally multi-scale [41]. Some existing work [30][20] [16] shows that considering the multi-scale characteristics can improve the performance of image quality assessment.…”

Section: Multi-scale Quality Fusion Strategymentioning

confidence: 99%

A Deep Learning based No-reference Quality Assessment Model for UGC Videos

Sun¹,

Min²,

Lü³

et al. 2022

Preprint

View full text Add to dashboard Cite

Quality assessment for User Generated Content (UGC) videos plays an important role in ensuring the viewing experience of end-users. Previous UGC video quality assessment (VQA) studies either use the image recognition model or the image quality assessment (IQA) models to extract frame-level features of UGC videos for quality regression, which are regarded as the sub-optimal solutions because of the domain shifts between these tasks and the UGC VQA task. In this paper, we propose a very simple but effective UGC VQA model, which tries to address this problem by training an end-to-end spatial feature extraction network to directly learn the quality-aware spatial feature representation from raw pixels of the video frames. We also extract the motion features to measure the temporal-related distortions that the spatial features cannot model. The proposed model utilizes very sparse frames to extract spatial features and dense frames (i.e. the video chunk) with a very low spatial resolution to extract motion features, which thereby has low computational complexity. With the better quality-aware features, we only use the simple multilayer perception layer (MLP) network to regress them into the chunk-level quality scores, and then the temporal average pooling strategy is adopted to obtain the video-level quality score. We further introduce a multi-scale quality fusion strategy to solve the problem of VQA across different spatial resolutions, where the multi-scale weights are obtained from the contrast sensitivity function of the human visual system. The experimental results show that the proposed model achieves the best performance on five popular UGC VQA databases, which demonstrates the effectiveness of the proposed model. The code will be publicly available.

show abstract

Section: Multi-scale Quality Fusion Strategymentioning

confidence: 99%

A Deep Learning based No-reference Quality Assessment Model for UGC Videos

Sun¹,

Min²,

Lü³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We employ temporal band-pass transforms inspired by similar successes [23], [24], [38], [67], [68] capturing temporal distortions. Temporal band-pass coefficients demonstrate reliable statistical regularities on very high quality videos, but these are predictably disturbed by the presence of distortions.…”

Section: Multiscale Learning and Temporal Transformationsmentioning

confidence: 99%

“…VFR-VQA is a challenging problem, since quality predictors must account for subtle perceptual quality changes occuring along the temporal dimension due to frame rate changes. As comparisons we also included FAVER [68], which is a current SOTA no-reference VFR-VQA model. From Table V, it may be observed that CONVIQT achieved competitive performance when compared against other NR-VQA models, indicating the frame rate discrimination capability of CONVIQT representations.…”

Section: Performance Comparison On Variable Frame Rate Videosmentioning

confidence: 99%

CONVIQT: Contrastive Video Quality Estimator

Madhusudana¹,

Birkbeck²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Perceptual video quality assessment (VQA) is an integral component of many streaming and video sharing platforms. Here we consider the problem of learning perceptually relevant video quality representations in a self-supervised manner. Distortion type identification and degradation level determination is employed as an auxiliary task to train a deep learning model containing a deep Convolutional Neural Network (CNN) that extracts spatial features, as well as a recurrent unit that captures temporal information. The model is trained using a contrastive loss and we therefore refer to this training framework and resulting model as CONtrastive VIdeo Quality EstimaTor (CONVIQT). During testing, the weights of the trained model are frozen, and a linear regressor maps the learned features to quality scores in a no-reference (NR) setting. We conduct comprehensive evaluations of the proposed model on multiple VQA databases by analyzing the correlations between model predictions and groundtruth quality ratings, and achieve competitive performance when compared to state-of-the-art NR-VQA models, even though it is not trained on those databases. Our ablation experiments demonstrate that the learned representations are highly robust and generalize well across synthetic and realistic distortions. Our results indicate that compelling representations with perceptual bearing can be obtained using self-supervised learning. The implementations used in this work have been made available at https://github.com/pavancm/CONVIQT.

show abstract

“…More recently, machine learning techniques have been employed in the development of perceptual metrics including VMAF [15], C3DVQA [17] and CONTRIQUE [65]. Alongside these generic quality metrics, assessment methods that were designed to specifically model the effect of frame rate/spatial resolution down-sampling or frame interpolation have been reported, including FRQM [66], ST-GREED [67], VSTR [68], FAVER [69] and FloLPIPS. However, none of these methods have been rigorously benchmarked due to the lack of databases with diverse content and ground-truth metadata.…”

Section: B Objective Quality Assessment For Vfimentioning

confidence: 99%

“…For completeness, we have also considered no-reference (NR) quality metrics. These include image models, NIQE [101] and deepIQA-NR [99], and video models, VIIDEO [102], VBLIINDS [103], VIDEVAL [104], RAPIQUE [105], and FAVER [69].…”

Section: Evaluation Of Objective Quality Metricsmentioning

confidence: 99%