Improving dynamic software analysis by applying grammar inference principles

Abstract

SUMMARYGrammar inference is a family of machine learning techniques that aim to infer grammars from a sample of sentences in some (unknown) language. Dynamic analysis is a family of techniques in the domain of software engineering that attempts to infer rules that govern the behaviour of software systems from a sample of executions. Despite their disparate domains, both fields have broadly similar aims; they try to infer rules that govern the behaviour of some unknown system from a sample of observations. Deriving general rules about program behaviour from dynamic analysis is difficult because it is virtually impossible to identify and supply a complete sample of necessary program executions. The problems that arise with incomplete input samples have been extensively investigated in the grammar inference community. This has resulted in a number of advances that have produced increasingly sophisticated solutions that are more successful at accurately inferring grammars from (potentially) sparse information about the underlying system. This paper investigates the similarities and shows how many of these advances can be applied with similar effect to dynamic analysis problems by a series of small experiments on random state machines.

Iterative Refinement of Reverse-Engineered Models by Model-Based Testing

“…Here we focus on the latter (dynamic) approaches. They are based on the analysis of program traces [5,6] which are sequences of events (e.g. function calls, messagepassing events etc.…”

“…This is akin to the challenge of inferring a regular grammar, which is conventionally represented as a state machine from a given set of strings (a problem originally posed in 1967 [7]). In fact, most reverse-engineering techniques are inspired by techniques that were initially devised as grammar-inference techniques [8,1,6].…”

“…An inference technique will only produce an accurate result if the provided set of traces is characteristic of the behaviour of the underlying software system [8,6]. In terms of state machines, this must include enough information about what the program can and cannot do to enable the inference technique to identify every state transition, and to distinguish between every pair of non-equivalent states.…”

“…Thus here we use state-merging techniques which place a greater emphasis on heuristics. These are less demanding in terms of the number of questions asked, but have nonetheless been shown to produce models that are reasonably accurate [8,6].…”

Incrementally Discovering Testable Specifications from Program Executions

“…As with most techniques that are based on program traces (referred to as dynamic analysis techniques), the selection of traces is critical [4]. One of the key novelties of our technique is that, instead of relying on the developer to record and supply traces, they are obtained automatically by using a model-based test-set generator.…”