Probabilistic and statistical temporal analyses have been developedas a means of determining the worst-case execution and responsetimes of real-time software for decades. A number of such methodshave been proposed in the literature, of which the majority claim tobe able to provide worst-case timing scenarios with respect to agiven likelihood of a certain value being exceeded. Further, suchclaims are based on either some estimates associated with a probability,or probability distributions with a certain level of confidence.However, the validity of the claims are very much dependent on anumber of factors, such as the achieved samples and the adopteddistributions for analysis.In this paper, we investigate whether the claims made are in facttrue as well as the establishing an understanding of the factors thataffect the validity of these claims. The results are of importancefor two reasons: to allow researchers to examine whether there areimportant issues that mean their techniques need to be refined; andso that practitioners, including industrialists who are currently usingcommercial timing analysis tools based on these types of techniques,understand how the techniques should be used to ensure theresults are fit for their purposes.
Study of the Reliability of Statistical Timing Analysis forReal-Time Systems
ABSTRACTProbabilistic and statistical temporal analyses have been developed as a means of determining the worst-case execution and response times of real-time software for decades. A number of such methods have been proposed in the literature, of which the majority claim to be able to provide worst-case timing scenarios with respect to a given likelihood of a certain value being exceeded. Further, such claims are based on either some estimates associated with a probability, or probability distributions with a certain level of confidence. However, the validity of the claims are very much dependent on a number of factors, such as the achieved samples and the adopted distributions for analysis. This paper is the first one that puts side by side existing state of the art statistical and probabilistic analysis techniques, using the probabilistic analysis as the ground truth in order to asses the applicability and performance of the statistical technique. The evaluation clearly shows that for the experiments performed the approach can identify clear differences between a range of techniques and that these differences can be considered valid based on the trends expected from the academic theory.