Advanced testing and debugging support for reactive executable DSLs

Abstract

Executable Domain-Specific Languages (xDSLs) allow the definition and the execution of behavioral models. Some behavioral models are reactive, meaning that during their execution, they accept external events and react by exposing events to the external environment. Since complex interaction may occur between the reactive model and the external environment, they should be tested as early as possible to ensure the correctness of their behavior. In this paper, we propose a set of generic testing facilities for reactive xDSLs using the standardized Test Description Language (TDL). Given a reactive xDSL, we generate a TDL library enabling the domain experts to write and run event-driven TDL test cases for conforming reactive models. To further support the domain expert, the approach integrates interactive debugging to help in localizing defects, and mutation analysis to measure the quality of test cases. We evaluate the level of genericity of the approach by successfully writing,

Automatic test amplification for executable models

“…An xDSL with testing support enables domain experts to test conforming behavioral models early in the design phase. In [28], we proposed a generic testing framework for xDSLs using the standard Test Description Language (TDL) [34]. Given an xDSL, the framework generates an xDSL-specific TDL library that provides facilities to define test cases and test data for the conforming models based on the xDSL definition, and then executing them and obtaining the result.…”

“…In the next section, we propose a generic test amplification approach for xDSLs. We base it on our earlier testing framework [28] since it supports the xDSLs considered in the context of this paper.…”

“…As the test input data are specific to the xDSL that the tested model conforms to, this component uses the xDSL definition for performing the modification. The new test cases are then given to the Test Runner (on the bottom left) which is the TDL test execution engine proposed in [28]. The engine executes the test cases on the model and produces their execution traces (label 2).…”

“…Testing is a popular dynamic V&V technique that involves executing systems and observing whether they act as expected. Currently, several testing approaches are proposed for xDSLs, some tailored for specific ones [25,29,33,37], while some others provide generic solutions that are applicable to a wide range of xD-SLs [10,27,28,36,54]. They allow the domain experts to write and execute test cases for behavioral models.…”

“…In this paper we propose a generic, automated approach for amplifying test suites of executable models. We focus on regression testing as a goal for test case generation, and use as a starting point our earlier testing framework [28], which supports the definition and execution of test cases for any executable model using the standard Test Description Language (TDL) [34]. Given an xDSL, a conforming behavioral model, and a TDL test suite for this model, our proposed approach amplifies this test suite in three steps.…”