Automatically Generating Test Cases for Specification Mining

Dallmeier, Valentin; Knopp, Nikolai; Mallon, Christoph; Fraser, Gordon; Hack, Sebastian; Zeller, Andreas

doi:10.1109/tse.2011.105

Cited by 62 publications

(58 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent research results [14,32,33] provide preliminary evidence that some of the existing inaccuracies may be circumvented by combining the different (state vs. sequence) types of execution information. However, previous efforts have combined this information in a limited manner.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Automatic mining of specifications from invocation traces and method invariants

Krka

Brun

Medvidović

2014

Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering

View full text Add to dashboard Cite

Software library documentation often describes the APIs of individual methods, but does not capture the intended protocols and method interactions. This can cause developers to misuse the library by invoking its methods out of sequence, while also restricting effective runtime detection of protocol violations and automated verification of the client code. Specification mining, if accurate, can help mitigate these problems. There are two classes of specification mining techniques: one that observes library executions, and another that identifies internal-state invariants. Unfortunately, models inferred by these state-of-the-art techniques often suffer from inaccuracies. In this paper, we aim to improve the quality of mined specifications in order to facilitate API understanding, test generation, and verification tasks. To that end, we combine the two classes of techniques in two novel ways, producing a pair of new model-inference techniques. We find that our new techniques improve significantly upon the state-of-the-art when applied to nine off-the-shelf libraries, increasing the precision of the inferred models by 4% and the recall by 41% on average. Additionally, we find that our approach automatically exposes both behavior that avoids raising exceptions and good-practice rules that capture proper library use, while being robust to noisy inputs.

show abstract

Section: Introductionmentioning

confidence: 99%

“…While the artifacts produced by these techniques can assist development tasks (e.g., test case generation [14,33] and debugging [4]), their utility critically depends on how close they are to the "true model." This, in turn, depends on the quality of the input executions, which are inherently partial.…”

Section: Introductionmentioning

confidence: 99%

Automatic mining of specifications from invocation traces and method invariants

Krka

Brun

Medvidović

2014

Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering

View full text Add to dashboard Cite

show abstract

“…Current approaches are "model-based", in the sense that they rely upon some reference model that can be used as a basis for computing accuracy [34,35,13,45]. This requirement for hand-crafted models can however be restrictive in terms of the size and complexity of the model against which a technique can be evaluated.…”

Section: K -Folds Cross Validationmentioning

confidence: 99%

“…As with all dynamic analysis techniques, an incomplete test set will yield an incomplete model [13]. However, it can be used in our experimental setting to compare the relative performance of different model inference configurations (this has previously been carried out by Lo et al [37]).…”

Section: K -Folds Cross Validationmentioning

confidence: 99%

Inferring extended finite state machine models from software executions

2015

View full text Add to dashboard Cite

The ability to reverse-engineer models of software behaviour is valuable for a wide range of software maintenance, validation and verification tasks. Current reverse-engineering techniques focus either on control-specific behaviour (e.g., in the form of Finite State Machines), or data-specific behaviour (e.g., as pre / post-conditions or invariants). However, typical software behaviour is usually a product of the two; models must combine both aspects to fully represent the software's operation. Extended Finite State Machines (EFSMs) provide such a model. Although attempts have been made to infer EFSMs, these have been problematic. The models inferred by these techniques can be non-deterministic, the inference algorithms can be inflexible, and only applicable to traces with specific characteristics. This paper presentsWe thank Ivo Krka at the University of Southern California for providing us with some of the traces for the evaluation. Ramsay Taylor and John Derrick are supported by the EU FP7 PROWESS projects. Neil Walkinshaw is grateful for the support provided by the UK Ministry of Defence (MOD) through the BATS and HASTE projects. Information contained in this document should not be interpreted as representing the views of the MOD, nor should it be assumed that it reflects any current or future MOD policy. The information cannot supersede any statutory or contractual requirements or liabilities and is offered without prejudice or commitment.

show abstract

“…The work in [13] presents an approach that, through a combination of systematic test case generation (by means of the TAUTOKO tool) and typestate mining, infers models of program behavior in the form of finite state automata describing transitions between object states. The generation of test cases permits to cover previously unobserved behavior, and systematically extends the execution space, and enriches the inferred behavior model.…”

Section: Related Workmentioning

confidence: 99%

Automated Integration of Service-Oriented Software Systems

Autili

Inverardi

Tivoli

2015

Fundamentals of Software Engineering

View full text Add to dashboard Cite

Abstract. In the near future we will be surrounded by a virtually infinite number of software applications that provide services in the digital space. This situation radically changes the way software will be produced and used: (i) software is increasingly produced according to specific goals and by integrating existing software; (ii) the focus of software production will be shifted towards reuse of third-parties software, typically blackbox, that is often provided without a machine readable documentation. The evidence underlying this scenario is that the price to pay for this software availability is a lack of knowledge on the software itself, notably on its interaction behaviour. A producer will operate with software artefacts that are not completely known in terms of their functional and non-functional characteristics. The general problem is therefore directed to the ability of interacting with the artefacts to the extent the goal is reached. This is not a trivial problem given the virtually infinite interaction protocols that can be defined at application level. Different software artefacts with heterogeneous interaction protocols may need to interoperate in order to reach the goal. In this paper we focus on techniques and tools for integration code synthesis, which are able to deal with partial knowledge and automatically produce correct-by-construction serviceoriented systems with respect to functional goals. The research approach we propose builds around two phases: elicit and integrate. The first concerns observation theories and techniques to elicit functional behavioural models of the interaction protocol of black-box services. The second deals with compositional theories and techniques to automatically synthesize appropriate integration means to compose the services together in order to realize a service choreography that satisfies the goal.

show abstract

Automatically Generating Test Cases for Specification Mining

Cited by 62 publications

References 28 publications

Automatic mining of specifications from invocation traces and method invariants

Automatic mining of specifications from invocation traces and method invariants

Inferring extended finite state machine models from software executions

Automated Integration of Service-Oriented Software Systems

Contact Info

Product

Resources

About