Adaptive Streaming: A subjective catalog to assess the performance of objective QoE metrics

Abstract

Scalable streaming has emerged as a feasible solution to resolve users' heterogeneity problems. Scalable Video Streaming (SVC) is the technology that has served as the definitive impulse for the growth of streaming adaptive systems. Systems need to incorporate user's Quality of Experience (QoE) in the layer switching algorithms. This paper characterizes the relation between human decisions and objective metrics that could be mapped into systems. We have performed extensive subjective experiments to corroborate the preference towards adaptive systems when compared to traditional non-adaptive systems. The resulting subjective scores are correlated with most relevant Full Reference (FR) objective metrics. We obtain an exponential relationship between human decisions and the same decisions expressed as a difference of objective metrics. A strong correlation with subjective scores validates objective metrics to be used as aid in the adaptive decision taking algorithms to improve overall systems performance. Results show that, among the evaluated objective metrics, PSNR is the metric that provide worse results in terms of reproducing the human decisions.

Autonomous video compression system for environmental monitoring

“…We also want to investigate the frequency with which, the execution of the algorithm must be executed to achieve the best balance between the improvements in the videos and the overload of the system. Because the final goal is to have an adaptive video streaming system while maintaining the QoE of the final user [14], as well as other authors have done for VoIP [15], we would like to improve the code implemented in Python and add more functionality in terms of codec decision taken, video conversion formats and the selection of capture device among others. Finally, it could be interesting to add, other parameters related to image properties such as luminance or exposure could be included.…”

Choosing the best video compression codec depending on the recorded environment

“…We also want to investigate the frequency with which the execution of the algorithm must be performed to achieve the best balance between improvements of videos and the system overhead due to the repeated execution of process. The final goal is to have adaptive video streaming while maintaining the QoE of the end user [15] as other authors have been done for VoIP [16].…”

Special Issue on JITEL 2013 Best Papers

“…Last but not least, the performance of objective QoE metrics for adaptive streaming services is studied in paper [7] co-authored by A. Álvarez et al In this work, the authors successfully perform extensive subjective experiments that show the user´s preference towards adaptive streaming systems. Built upon a found strong correlation with subjective scores, the authors introduce objective metrics in the adaptive streaming algorithm, in such a way that the overall system performance is improved.…”