Modbat: A Model-Based API Tester for Event-Driven Systems

Abstract

Software testing executes a system under test by giving it a series of inputs and comparing the outputs to expected values. Model-based testing generates test executions from an abstract model that describes the system behavior. Existing model-based approaches are not wellsuited for event-driven or input/output-driven systems. In particular, there is a need to support nonblocking I/O operations, or operations throwing exceptions when communication is disrupted. We present a new tool called "Modbat", which is specialized for testing the application programming interface of systems where these issues are common. * , JSPS kaken-hi 23240003.

Precondition Coverage in Software Testing

“…For object-oriented software, test cases are often designed to cover possible exceptions, but tend to stop at the first exception [8], [19]. This bias was confirmed in our case studies for designing models for network libraries [2], [3]. In this work, we present evidence for a bias for using valid parameters or states over invalid ones, which again shows that incorrect uses of libraries (including multiple incorrect uses) are not sufficiently considered by human testers.…”

“…We analyze the outcome of preconditions in the regression test suite of Scala 2.11.7 [15] and several transition-based test models written for Modbat [2]. We chose the Scala library because it offers Eiffel-like contracts that can also be used by Modbat models, which itself is based on Scala as well.…”

“…In addition to that, a transition function may optionally feature an enabling function (a precondition), which must hold for a transition to be executable. ModelJUnit [20] and Modbat [2] are two model-based testing tools that support such enabling functions.…”

Model-based Testing of the Java Network API

“…Instead of designing individual test cases, test models describe entire sets of possible tests. More test tools than can be described here exist, based on state machines [1,8,17] or constraint specifications [7,13]. Test models (as well as unit tests) are usually designed based on the specification [17].…”

“…Tools derive test sequences from a model that specifies possible test executions and expected results [1,7,8,13,17]. We use Modbat [1] because it offers an embedded domain-specific language [19] that combines extended finite-state machines [6] with low-level monitoring code written in Scala [14].…”