Coverage-biased random exploration of large models and application to testing

Abstract

Abstract. This article presents several related methods for drawing traces. First, it is shown how to draw traces uniformly at random in large models composed of several components. Then a method for drawing traces according to a given coverage criterion is presented, together with a notion of randomised coverage satisfaction. These methods rely on combinatorial algorithms, based on a representation of the model by an automaton or by a product of several automata, synchronised or not. We report several experimental results on random generation of traces in large transition systems, and on statistical testing of C programs.

Enumerative encoding of correlation-immune Boolean functions

“…The approach, based on finding recursive decompositions of the classes, is also called the recursive method in [19,12,11]. This is a general method, that can be applied each time all objects of size n can be partitioned into equivalence classes.…”

An Approximation-Based Approach for the Random Exploration of Large Models

“…As far as we know, the first work combining random testing and model-based testing has been proposed in [14] as a combination of model-checking and testing. In [9] the authors have proposed an improved approach to explore the models at random. This technique has been extended to pushdown models [15,11] and to grammar-based systems [10].…”

“…However, the nature of random testing is to draw randomly a test rather than choosing it, and it is therefore inefficient to detect behaviour of a program occurring with a very low probability. In [9], a random testing approach consisting of the exploration of large graph based models has been proposed. In order to tackle the problem of low probabilistic behaviour, the authors have also suggested to bias the random generation, by combining it with a coverage criterion, in order to optimize the probability to meet system' features described by this criterion.…”

Random Grammar-Based Testing for Covering All Non-terminals

“…The general schema for this combination, as described in [1], is the following: considering a random generation algorithm of test data of size n and a coverage criteria C (each element of C is or is not covered by each possible test), the goal is to use the generation algorithm N times in order to optimise the probability of covering all elements of C. For each element e ∈ C, we denote by p e,n the probability that a generated test of size n covers e. One can easily check that generating N test data independently of C provides a probability of covering C of 1−(1−p min ) N , where p min = min e∈C {p e,n }. This probability is the way to measure the quality of the testing approach, relatively to C. A better way is to repeat N times the following procedure:…”

“…In [1], it is explained how to bias a uniform random testing approach using constraints given by a coverage criterion, in order to optimise the probability of satisfying this criterion. The technique is developed for path generation in a graph.…”