Self-Diagnosing Low Coverage and High Interference in 3G/4G Radio Access Networks based on Automatic RF Measurement Extraction

A No-Reference Video Streaming QoE Estimator based on Physical Layer 4G Radio Measurements

“…• For all ML algorithms, the majority of the performance metrics indicates that the unbalanced training set produces the best results; however, from the Stratified Error metric, it is visible that there is a decrease in the MSE from the unbalanced to the balanced training sets, in the intervals of ]2, 3] and ]3, 4], and only a marginal increase in the interval [1,2] for ET and AB algorithms. Additionally, an increase in MSE is verified from the unbalanced to the balanced training sets in the ]4,5] stratum which is not significant since it is more important to accurately predict the MOS in the lower strata.…”

“…In order to address the concerns posed at the end of Section II-A regarding the possible learning impairments when the ML algorithms receive as input a severely unbalanced dataset, the initial training set was modified so that all MOS strata were more evenly represented in the resulting distribution. To test the hypothesis that the prediction accuracy of the more critical values of MOS -those belonging to the strata [1,2], ]2, 3]could increase, 90% of the sessions belonging to the interval ]4,5] were removed, at random, from the training set, in order to achieve a more balanced training set while still capturing the tendency, from the original distribution, of the highest MOS interval of being the most represented. Ten models were then created using the unbalanced and balanced training sets, and the five aforementioned ML algorithms; the models prediction performance was analyzed using the metrics listed in Section III-A, and the results are presented in Table I.…”

“…Additionally, an increase in MSE is verified from the unbalanced to the balanced training sets in the ]4,5] stratum which is not significant since it is more important to accurately predict the MOS in the lower strata. The decrease in MSE, seen in the intervals ]2, 3], ]3, 4] for all algorithms and [1,2] for the GTB, RF and SVR algorithms, proves that when a ML algorithm uses a balanced training set, it can produce more accurate predictions for less represented values than when using an unbalanced training set; • All considered ML algorithms could be used, with varying degrees of performance and complexity, to obtain acceptable MOS predictions; • Both the GTB and RF have similar performances when using the balanced training set in terms of the Stratified Error, and are able to outperform the ET and AB algorithms in this particular metric for the two lowest intervals of MOS. Furthermore, the GTB algorithm outperforms the RF in eight of the performance metrics, being only marginally worse in two metrics.…”

“…One of the most prominent issues that current and nextgeneration wireless network operators will face is assuring high Quality of Experience (QoE) to an increasing number of users, in services that generate a high network load, as video streaming [1]. In this context, much work has been done on assessing Quality of Service (QoS) through the measurement of specific network key performance indicators (KPIs) and drive tests (DTs) (e.g., [2] [3] [4]). However, these metrics cannot directly assess the user's QoE.…”

“…and Pearson correlations were used to assess the relationship between predicted and ground truth MOS values;• 10-Fold Cross Validation (CV) using Mean Squared Stratified Error -By splitting the dataset in four MOS strata (i.e.,[1,2], ]2, 3], ]3, 4], ]4, 5]), the MSE can be computed per stratum, allowing a higher granularity for the error analysis. In fact, due to the skewed MOS distribution towards the highers values, if the MSE is measured on the whole MOS range it will be mainly influenced by the error in the higher MOS values, preventing a proper assessment of the model performance in the lower MOS range.…”