Video Quality of Experience Metric for Dynamic Adaptive Streaming Services Using DASH Standard and Deep Spatial-Temporal Representation of Video

Abstract

DASH (Dynamic Adaptive Streaming over HTTP (HyperText Transfer Protocol)) as a universal unified multimedia streaming standard selects the appropriate video bitrate to improve the user’s Quality of Experience (QoE) according to network conditions, client status, etc. Considering that the quantitative expression of the user’s QoE is also a difficult point in itself, this paper researched the distortion caused due to video compression, network transmission and other aspects, and then proposes a video QoE metric for dynamic adaptive streaming services. Three-Dimensional Convolutional Neural Networks (3D CNN) and Long Short-Term Memory (LSTM) are used together to extract the deep spatial-temporal features to represent the content characteristics of the video. While accounting for the fluctuation in the quality of a video caused by bitrate switching on the QoE, other factors such as video content characteristics, video quality and video fluency, are combined to form the input feature vector. The ridge regression method is adopted to establish a QoE metric that enables to dynamically describe the relationship between the input feature vector and the value of the Mean Opinion Score (MOS). The experimental results on different datasets demonstrate that the prediction accuracy of the proposed method can achieve superior performance over the state-of-the-art methods, which proves the proposed QoE model can effectively guide the client’s bitrate selection in dynamic adaptive streaming media services.

A Survey on Multimedia Services QoE Assessment and Machine Learning-Based Prediction

“…The predictive models for DASH video streaming [133]- [135] take into consideration the technical characteristics and network's conditions that impacting video quality and consequently QoE. The assessment of QoE relies on subjective metrics such as MOS and ACR [134], [135], as well as objective metrics including the FR MS-SSIM, the RR STRRED and the NR NIQE metric [133]. The highest prediction accuracy is achieved through the utilization of a model based on a combination of RNN and long short-term memory (LSTM) algorithms [133], which succeeds to reflect the nonlinearity and complicated temporal dependence owing to adaptive streaming speed adjustments of QoE.…”

“…The highest prediction accuracy is achieved through the utilization of a model based on a combination of RNN and long short-term memory (LSTM) algorithms [133], which succeeds to reflect the nonlinearity and complicated temporal dependence owing to adaptive streaming speed adjustments of QoE. In [134], a QoE video DASH metric approach is presented that relies on three-dimensional convolutional neural networks (3D CNN) and LSTM, and utilizes the ridge regression technique to provide a QoE metric, which dynamically describes the correlation among the input characteristics vector and the MOS value. In [135], adaptive bitrate streaming (ABS) algorithms are analyzed, and an ML model based on decision tree regression (DTR), multi-linear regression (MLR) and random forest regression (RFR) is provided to evaluate QoE in DASH video streaming with respect to network metrics.…”

Audio-Visual Quality of Experience Prediction Based on ELM Model

“…The following equation is used to convert data back into unnormalize units: Xuni=Xni×(max (Xi)-(min (Xi) +min (Xi), i=1,2,3...n (5) Yunp=Ynp× (max (Yp)min (Yp) +min (Yp), p=1,2…n (6) Where: Xuni: i th unnormalized input value in the dataset. Xni : i th input value in the dataset.…”

“…The QoE is the degree of end-user satisfaction that considers all the elements that impact it [4]. The QoE is a critical measure that network operators and service providers can utilize to assess their performance by considering all elements that influence it [5].…”

Unsupervised Learning Data-Driven Continuous QoE Assessment in Adaptive Streaming-Based Television System

“…Additionally, 70.83% of test subjects answered that stalls are the most relevant metric when evaluating the quality of the video. 16.67% of users considered quality the most relevant aspect [9]. The Hammerstein-Wiener predictor has been used to create a QoE evaluator called time-varying QoE Indexer, which accounts for interactions between stalls, analyzes video content and perceptual video quality, and predicts continuous-time QoE.…”