Video Quality Analysis: Steps towards Unifying Full and No Reference Cases

Abstract

Video quality assessment (VQA) is now a fast-growing field, maturing in the full reference (FR) case, yet challenging in the exploding no reference (NR) case. In this paper, we investigate some variants of the popular FR VMAF video quality assessment algorithm, using both support vector regression and feedforward neural networks. We also extend it to the NR case, using different features but similar learning, to develop a partially unified framework for VQA. When fully trained, FR algorithms such as VMAF perform very well on test datasets, reaching a 90%+ match in the popular correlation coefficients PCC and SRCC. However, for predicting performance in the wild, we train/test them individually for each dataset. With an 80/20 train/test split, we still achieve about 90% performance on average in both PCC and SRCC, with up to 7–9% gains over VMAF, using an improved motion feature and better regression. Moreover, we even obtain good performance (about 75%) if we ignore the reference, treating FR as NR, partly justifying our attempts at unification. In the true NR case, typically with amateur user-generated data, we avail of many more features, but still reduce complexity vs. recent algorithms VIDEVAL and RAPIQUE, while achieving performance within 3–5% of them. Moreover, we develop a method to analyze the saliency of features, and conclude that for both VIDEVAL and RAPIQUE, a small subset of their features provide the bulk of the performance. We also touch upon the current best NR methods: MDT-VSFA, and PVQ which reach above 80% performance. In short, we identify encouraging improvements in trainability in FR, while constraining training complexity against leading methods in NR, elucidating the saliency of features for feature selection.

ASViS: A Protocol for Efficient Video Streaming Using SVC and Deadline-based Criteria.

“…Defining video quality is a contentious issue since it depends on several subjective factors, and replicating subjectivity as an objective measure is challenging [23,24]. Focusing on the human eye and its biological process of converting visible light into visual information, we encounter an essential factor, the photoreceptors [25].…”

“…Therefore, they developed VMAF [24,29], a video quality metric capable of accurately capturing human perception. VMAF obtains a perceptual quality score by analyzing video quality using different approaches and then combining them using a support vector machine (SVM) [23,30]. The combination approach helps to preserve the contribution of each individual measure and increase its correlation with subjective ratings.…”

Adaptive Scalable Video Streaming (ASViS): An Advanced ABR Transmission Protocol for Optimal Video Quality

“…Determining video quality has long been debated due to its reliance on numerous subjective factors. The challenge lies in transforming these subjective perceptions into objective metrics [24,25]. Taking into account the human eye process of translating visible light into visual data, the prominence of photoreceptors becomes apparent [26].…”

“…In response to these limitations, Netflix introduced VMAF, a video quality metric crafted to mirror human perception more accurately [25,30]. By amalgamating various video quality evaluation methods using a Support Vector Machine (SVM), VMAF produces a perceptual quality score [12,24,31]. This integrated approach ensures the holistic representation of individual measures, bolstering its correlation with subjective assessments.…”