Measurement-Based Worst-Case Execution Time Estimation Using the Coefficient of Variation

Abstract

Extreme Value Theory (EVT) has been historically used in domains such as finance and hydrology to model worst-case events (e.g. major stock market incidences). EVT takes as input a sample of the distribution of the variable to model and fits the tail of that sample to either the Generalised Extreme Value (GEV) or the Generalised Pareto Distribution (GPD). Recently, EVT has become popular in real-time systems to derive worst-case execution time (WCET) estimates of programs. However, the application of EVT is not straightforward and requires a detailed analysis of, and customisation for, the particular problem at hand. In this paper we tailor the application of EVT to timing analysis. To that end (1) we analyse the response time of different hardware resources (e.g. cache memories) and identify those that may lead to radically different types of execution time distributions. (2) We show that one of these distributions, known as mixture distribution, causes problems in the use of EVT. In particular, mixture distributions challenge not only properly selecting GEV/GPD parameters (i.e., location, scale and shape), but also determining the size of the sample to ensure that enough tail values are passed to EVT and that only tail values are used by EVT to fit GEV/GPD. Failing to select these parameters has a negative impact on the quality of the derived WCET estimates. We tackle these problems, by (3) proposing MBPTA-CV, a new mixture-distribution aware, WCET-suited MBPTA method that builds upon recent EVT developments in other fields (e.g. finance) to automatically select the distribution parameters that best fit the maxima of the observed execution times. Our results on a simulation environment and a real board show that MBPTA-CV produces high-quality WCET estimates.

Measurement-based cache representativeness on multipath programs

“…Observation 2: e pWCET curve obtained from the sample of a pETd is (probabilistically) an upper-bound for the execution time distribution that has been sampled [2].…”

“…$15.00 DOI: 10.1145/3195970.3196075 e increased variability in the execution time of a task, whose distribution can have arbitrary variance and shape, has motivated several research works to use statistical techniques to derive bounds to execution time distributions. is has resulted in a variant of MBTA referred to as Measurement-based probabilistic timing analysis (MBPTA) [2]. MBPTA delivers a probabilistic WCET (pWCET) function (see Figure 1(a)) upper-bounding the execution time distribution (pETd) of the program at any exceeding probability.…”

“…MBPTA [2] applies Extreme Value eory (EVT) [6,16] on a set of execution time measurements, which must meet certain statistical properties (e.g. independence and identical distribution), and determines the best set of maxima values of the sample to be used to estimate the pWCET .…”

On uses of extreme value theory fit for industrial-quality WCET analysis

“…Yet, for the argument we have given above (further developed in [23]), the extreme behaviour of the execution-time distribution of a real-time software program can always be modelled with a Gumbel distribution: it may not be the tightest one, but it is always -at least -a safe over-approximation of it. Unless we assume that our program of interest has an unbounded WCET, using the Fréchet distribution should be considered inappropriate.…”

“…If a Fréchet distribution were to best fit a given sample of execution-time measurements, this should happen because either the sample does not contain enough tail values or some of them do not really belong to the tail of the distribution. The remedy would be to increase the sample (to capture more tail values) or use different BM block sizes (to discard less tail values), respectively, as illustrated in [23].…”