One evaluation of model-based testing and its automation

Abstract

Model-based testing relies on behavior models for the generation of model traces: input and expected output-test cases-for an implementation. We use the case study of an automotive network controller to assess different test suites in terms of error detection, model coverage, and implementation coverage. Some of these suites were generated automatically with and without models, purely at random, and with dedicated functional test selection criteria. Other suites were derived manually, with and without the model at hand. Both automatically and manually derived model-based test suites detected significantly more requirements errors than handcrafted test suites that were directly derived from the requirements. The number of detected programming errors did not depend on the use of models. Automatically generated model-based test suites detected as many errors as hand-crafted model-based suites with the same number of tests. A sixfold increase in the number of modelbased tests led to an 11% increase in detected errors.

An Activity-Based Quality Model for Maintainability

“…Moreover, the general method of modeling -that inherently includes structuring -improved the guidelines: although the initial MAN guideline included many important aspects, we still were able to reveal several omissions and inconsistencies. Building the model, similar to other model building activities in software engineering [23], revealed these problems and allowed to solve them.…”

An Object-Oriented Framework for Improving Software Reuse on Automated Testing of Mobile Phones

“…As an example, model-based testing is an approach in which the behavior of the software under test is described by means of formal models (such as Petri nets, state charts and so forth) as a starting point to automatic or semi-automatic TC generation [3] [4] [5] [6]. Other approach relies on algorithms able to create test cases which cover the interaction among different applications in rich-feature communicating systems [7].…”

On the Complexity of Input Output Conformance Testing

“…Testing is a major part of the software development process and, together with debugging, accounts for more than half of the development cost and effort [14]. Model-based testing is a structured and rigorous discipline of testing, which is likely to improve the current practice of testing [24,23,17]. Input-output conformance (ioco) testing is a well-known formal approach to model-based testing, which is used extensively in various practical applications, see [13,5] and the references therein, and which has been the subject of much theoretical research, see [21] and the references therein.…”