Learning a Behavior Model of Hybrid Systems Through Combining Model-Based Testing and Machine Learning

Abstract

Models play an essential role in the design process of cyberphysical systems. They form the basis for simulation and analysis and help in identifying design problems as early as possible. However, the construction of models that comprise physical and digital behavior is challenging. Therefore, there is considerable interest in learning such hybrid behavior by means of machine learning which requires sufficient and representative training data covering the behavior of the physical system adequately. In this work, we exploit a combination of automata learning and model-based testing to generate sufficient training data fully automatically. Experimental results on a platooning scenario show that recurrent neural networks learned with this data achieved significantly better results compared to models learned from randomly generated data. In particular, the classification error for crash detection is reduced by a factor of five and a similar F1-score is obtained with up to three orders of magnitude fewer training samples.

Runtime Monitors for Markov Decision Processes

“…The monitoring problem for MDPs then conservatively over-approximates the risk of a trace by assuming an adversarial scheduler, that is, by taking the supremum risk estimate over all schedulers 1 . The Monitoring Problem.…”

“…Predictive monitoring has been combined with deep learning [17] and Bayesian inference [22], where the key problem is that the computation of an imminent failure is too expensive to be done exactly. More generally, learning automata models has been motivated with runtime assurance [1,55]. Testing approaches statistically evaluate whether traces are likely to be produced by a given model [25].…”

Interoperability and Integration Testing Methods for IoT Systems: a Systematic Mapping Study

“…Regarding established testing techniques, path-based testing using finite state machines (or analogous structures) as a system under test (SUT) model is discussed in five studies [55,7,5,25,20]. In addition, an SUT model based on a timed state machine has also been employed [35].…”

“…The MBT approach, in general, is explicitly discussed in several studies [2,1,29,9,5,36,25,20], which mostly describe a general concept; particular testing technique, namely path-based testing, is discussed in the studies by Aicherning et al [5], Estivill-Castro et al [20], and Grace et al [25].…”

Automata Learning for Automated Test Generation of Real Time Localization Systems

“…An interpretable model generates knowledge about a system's functionality and allows the generation of complete, application-specific sets of test cases. We use the automaton model to perform an automated test case generation [8].…”