On Active Sampling of Controlled Experiments for QoE Modeling

Abstract

For internet applications, measuring, modeling and predicting the quality experienced by end users as a function of network conditions is challenging. A common approach for building application speci c Quality of Experience (QoE) models is to rely on controlled experimentation. For accurate QoE modeling, this approach can result in a large number of experiments to carry out because of the multiplicity of the network features, their large span (e.g., bandwidth, delay) and the time needed to setup the experiments themselves. However, most often, the space of network features in which experimentations are carried out shows a high degree of uniformity in the training labels of QoE. This uniformity, di cult to predict beforehand, ampli es the training cost with little or no improvement in QoE modeling accuracy. So, in this paper, we aim to exploit this uniformity, and propose a methodology based on active learning, to sample the experimental space intelligently, so that the training cost of experimentation is reduced. We prove the feasibility of our methodology by validating it over a particular case of YouTube streaming, where QoE is modeled both in terms of interruptions and stalling duration.

From Network Traffic Measurements to QoE for Internet Video

“…Here a machine learning classification model is used to intelligently select network instances for experimentation. In a prior work [16], we have shown that active learning gives a significant gain over uniform sampling in QoS-QoE modeling. However, active sampling requires a single output QoE definition (classification label) to be predefined before the experimentation phase, making the resulting dataset biased towards the given classification model.…”

ACQUA: A user friendly platform for lightweight network monitoring and QoE forecasting

“…Indeed, YouTube QoE is known to decrease fast with the join time and the duration of stalls. Explicit functions exist in the literature to capture this dependence, further details in [7]. We stream thousands of videos in variable network conditions, then for each streaming session, we measure the join time and the duration of stalls, which together give us an estimation of the QoE for this session on a scale from 1 to 5.…”

“…With this approach, we aim at making network measurements reused by different QoE models and allowing ACQUA to provide estimations of the QoE without running the applications themselves, hence reducing the overhead on the mobile. The way we build these models and their accuracy have been discussed in two of our recent papers for two popular applications, Skype [6] and YouTube [7].…”

“…As said above, the models of QoE in ACQUA are built offline with the help of controlled experimentation [6] [7]. The network conditions are artificially tuned using network emulation tools, applications are played out, and the corresponding QoE is written down.…”

An Online Sampling Approach for Controlled Experimentation and QoE Modeling

“…We have applied pool based uncertainty sampling in a previous work [9] where we presented a first validation with a trace collected and labeled with uniform sampling, thus forming our ground truth. With the help of a video streaming QoE case, we validated the gain of uncertainty sampling and its capacity to reduce the training cost by an order of magnitude.…”