Increasing Functional Coverage by Inductive Testing: A Case Study

Abstract

Abstract. This paper addresses the challenge of generating test sets that achieve functional coverage, in the absence of a complete specification. The inductive testing technique works by probing the system behaviour with tests, and using the test results to construct an internal model of software behaviour, which is then used to generate further tests. The idea in itself is not new, but prior attempts to implement this idea have been hampered by expense and scalability, and inflexibility with respect to testing strategies. In the past, inductive testing techniques have tended to focus on the inferred models, as opposed to the suitability of the test sets that were generated in the process. This paper presents a flexible implementation of the inductive testing technique, and demonstrates its application with case-study that applies it to the Linux TCP stack implementation. The evaluation shows that the generated test sets achieve a much better coverage of the system than would be achieved by similar non-inductive techniques.

Case Studies in Learning-Based Testing

“…However, this approach is more focussed on finding untrapped exceptions then testing formal user requirements (model checking is not used). However, [16] confirms our own findings that learning algorithms give more efficient search methods than random testing.…”

“…Inductive testing (Walkinshaw et al [16]) is a black-box testing technique that also uses automata learning algorithms. However, this approach is more focussed on finding untrapped exceptions then testing formal user requirements (model checking is not used).…”

Inferring extended finite state machine models from software executions

“…So far, this has concentrated solely on simple finite state machines [48,49,42] and data classifiers [18]. It is envisaged that the ability to incorporate these richer models will lead the the ability to produce more more rigorous test sets.…”

Behaviourally Adequate Software Testing

“…If on the other hand a cheaper inference process is adopted that allows for an inexact model (c.f. previous work by Walkinshaw et al [13]), there has been no reliable means by which to gauge the accuracy of the final model, and to assess the adequacy of the final test set.…”

“…Since then, the idea of combining model inference with software testing has been comprehensively explored in several theoretical and practical contexts [8], [6], [7], [9], [10], [11], [12], [13]. Much of this work has focussed on the appealing, complementary relationship between program testing and machine learning.…”