Unbiased Benchmarking in Mobile Networks: The Role of Sampling and Stratification

Abstract

Cellular operators tightly monitor their networks to keep up with the market demand and frequently benchmark their performance against competitors. Typical benchmarking tests compare key performance indicators, quality of service, and quality of experience parameters on the city-and regional levels using user-collected crowdsourced data or drive test measurements. However, time-variant parameters and different user mobility patterns can bias the performance comparison. Designing a measurement sampling strategy that deals with such issues is critical for achieving a valid benchmark. Whether we would like to determine how many tiles of a map have to be measured in drive tests or how many samples we need from crowdsourced data to reach an estimate with the required accuracy, sampling theory can provide us with an answer. Since propagation conditions depend on user mobility and measurement environment, splitting the data set into groups or strata allows us to attain an unbiased estimate with fewer samples, thus allowing for a fair comparison to other mobile network operators with minimum effort measurements. In this work, we characterize the performance of different sampling methods on the simulated data set while investigating specific use cases to reveal scenarios where the stratification method pays off. We further analyze the sampling methods on two real-world crowdsourced data sets from a major Austrian operator. By stratifying the data into meaningful strata, we obtain the required number of areas and measurements in each stratum while remaining under the pre-set estimation error level. To our knowledge, this is the first study on sampling methodologies applied to real-world crowdsourced cellular measurements.

A Differentiable Throughput Model for Load-Aware Cellular Network Optimization Through Gradient Descent

“…Here, the promise of model-free methods is to directly include monitoring data into the optimization process, most notably from MDT, which plays an increasingly important role in the operation of cellular networks [22], [23]. In [2], the authors successfully utilize such MDT data for automated tuning of the antenna downtilt based on a RL approach.…”

“…Hence, our simulations are well validated for the considered region and make use of a detailed 3D building model of the environment to capture the relevant propagation mechanisms in urban areas. To adequately describe indoor locations, we further introduce an additional 10 dB penetration loss, which is in line with extensive MDT data analysis in [23]. We further use a constant noise term of ζ = −120 dBm, and set N RB = 100 with B RB = 180 kHz according to the LTE standard definition for a 20 MHz carrier.…”