Test oracles for simulink-like models

Abstract

The design of embedded systems is often supported by the definition of executable models for tools like Matlab/Simulink or Scilab/Xcos. These models play a pivotal role in the development process and their correctness is thus extremely important. Many different solutions exist for the definition of suitable tests to â\u80\u9cexerciseâ\u80\u9d these models, but only a few (partial) solutions exist for assessing the quality of execution (simulation) results, that is, for defining suitable oracles. This paper addresses the problem and proposes a formal language for specifying the oracles and relating them to existing models. It also presents Apolom, a prototype tool for checking simulation results against stated oracles. The empirical assessment we conducted to assess the viability of the proposed solution is organized around four case studies and witnesses interesting results in terms of effectiveness, efficiency, and required resources

Generating automated and online test oracles for Simulink models with continuous and uncertain behaviors

“…The CPS model is described in Simulink ( 1 ). Simulink is used by more than 60% of engineers for simulation of CPS [25,73], and is the prevalent modeling language in the automotive domain [53,72]. Simulink appeals to engineers since it is particularly suitable for specifying dynamic systems, and further, it is executable and allows engineers to test their models as early as possible.…”

Test orales for systems with complex outputs: the case of TTS systems

“…This would allow one to affirm that test activities are highly dependent on the SUT. As a consequence of this dependence and particularity of the SUTs, many software companies allocate a specific team of practitioners to develop their own testing tools (BARESI;YOUNG, 2001). …”

“…to verify whether the SUT behaves correctly on a particular execution (HOWDEN, 1978;BARESI;YOUNG, 2001;HASHIM, 2009). This element plays a decisive role in software testing processes establishing a decision about the correctness of an SUT result (WEYUKER, 1982;DAVIS;WEYUKER, 1981;DAY;GANNON, 1985).…”

“…On the other hand, even a human being can play the role of test oracle deciding about the correctness of test results. Regardless of the different ways of implementation, oracles bear a fundamental responsibility for the whole testing process: they judge and report to testers whether the result of an execution is correct (HOFFMAN, 1998;HOFFMAN, 2001;BARESI;YOUNG, 2001).…”

“…Different authors (BEIZER, 1990;HOFFMAN, 1998;BARESI;YOUNG, 2001;HOFF-MAN, 2006;HOFFMAN, 2001;WHALEN;HEIMDAHL, 2011;HARMAN et al, 2013) have dedicated their efforts in writing theories and classifications and taxonomies of test oracles. Due to the diversity of software domains, currently there is no standardized test oracle taxonomy and the existing classifications could depend on the source of information, the SUT's output or the automation method considered by the oracle.…”