Research and Improvements on Mutation Operators for Simulink Models

Yin, Yong; Zhou, Yi; Wang, Yan Rong

doi:10.4028/www.scientific.net/amm.687-691.1389

Cited by 4 publications

(6 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We developed a comprehensive list of Simulink fault patterns. We identified these patterns through our discussions with senior engineers from Delphi Automotive and by reviewing the existing literature on mutation operators for Simulink models [120], [17], [14], [114]. Tables 2 and 3 report these fault patterns.…”

Section: Experiments Designmentioning

confidence: 99%

See 1 more Smart Citation

Test Generation and Test Prioritization for Simulink Models with Dynamic Behavior

Matinnejad

Nejati

Briand

et al. 2019

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

All engineering disciplines are founded and rely on models, although they may differ on purposes and usages of modeling. Among the different disciplines, the engineering of Cyber Physical Systems (CPSs) particularly relies on models with dynamic behaviors (i.e., models that exhibit time-varying changes). The Simulink modeling platform greatly appeals to CPS engineers since it captures dynamic behavior models. It further provides seamless support for two indispensable engineering activities: (1) automated verification of abstract system models via model simulation, and (2) automated generation of system implementation via code generation. We identify three main challenges in the verification and testing of Simulink models with dynamic behavior, namely incompatibility, oracle and scalability challenges. We propose a Simulink testing approach that attempts to address these challenges. Specifically, we propose a black-box test generation approach, implemented based on meta-heuristic search, that aims to maximize diversity in test output signals generated by Simulink models. We argue that in the CPS domain test oracles are likely to be manual and therefore the main cost driver of testing. In order to lower the cost of manual test oracles, we propose a test prioritization algorithm to automatically rank test cases generated by our test generation algorithm according to their likelihood to reveal a fault. Engineers can then select, according to their test budget, a subset of the most highly ranked test cases. To demonstrate scalability, we evaluate our testing approach using industrial Simulink models. Our evaluation shows that our test generation and test prioritization approaches outperform baseline techniques that rely on random testing and structural coverage.

show abstract

Section: Experiments Designmentioning

confidence: 99%

“…We then generated 10,000 test inputs for each of the CPC and FPC models using the adaptive random testing algorithm. We executed each mutant candidate of CPC and FPC using the [120], [17], [14], [114] Fault Pattern.…”

Section: Experiments Designmentioning

confidence: 99%

Test Generation and Test Prioritization for Simulink Models with Dynamic Behavior

Matinnejad

Nejati

Briand

et al. 2019

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…Examples of fault patterns include incorrect signal data type, incorrect math operation, and incorrect transition condition. We identified these patterns through our discussions with senior Delphi engineers and by reviewing the existing literature on mutation operators for Simulink models [61,9,7,58]. We have developed an automated fault seeding program to automatically generate 44 faulty versions of CPC, 30 faulty versions of FPC, 17 faulty versions of CC, and 13 faulty versions of CLC (one fault per each faulty model).…”

Section: Experiments Designmentioning

confidence: 99%

Automated test suite generation for time-continuous simulink models

Matinnejad

Nejati

Briand

et al. 2016

Proceedings of the 38th International Conference on Software Engineering

View full text Add to dashboard Cite

All engineering disciplines are founded and rely on models, although they may differ on purposes and usages of modeling. Interdisciplinary domains such as Cyber Physical Systems (CPSs) seek approaches that incorporate different modeling needs and usages. Specifically, the Simulink modeling platform greatly appeals to CPS engineers due to its seamless support for simulation and code generation. In this paper, we propose a test generation approach that is applicable to Simulink models built for both purposes of simulation and code generation. We define test inputs and outputs as signals that capture evolution of values over time. Our test generation approach is implemented as a meta-heuristic search algorithm and is guided to produce test outputs with diverse shapes according to our proposed notion of diversity. Our evaluation, performed on industrial and public domain models, demonstrates that: (1) In contrast to the existing tools for testing Simulink models that are only applicable to a subset of code generation models, our approach is applicable to both code generation and simulation Simulink models. (2) Our new notion of diversity for output signals outperforms random baseline testing and an existing notion of signal diversity in revealing faults in Simulink models. (3) The fault revealing ability of our test generation approach outperforms that of the Simulink Design Verifier, the only testing toolbox for Simulink.

show abstract

“…Then we replace the normal input block to a switch block for each boolean input such that if the input value is less than 0.5, the output of the switch block will be 0, and otherwise, the output will be 1. TABLE 3: Simulink Fault Patterns Collected in Literature [51], [52], [53], [54], and also listed in [6].…”

Section: Case Studiesmentioning

confidence: 99%

What Not to Test (For Cyber-Physical Systems)

Ling

2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Testing complex Cyber Physical Systems (CPSs) can be expensive and time consuming. Current state-of-the-art methods that explore this problem are fully-supervised; i.e. they require that all examples are labeled. On the other hand, the GenClu system (introduced in this paper) takes a semi-supervised approach; i.e. (a) only a small subset of information is actually labeled (via simulation) and (b) those labels are then spread across the rest of the data. When applied to five open-source CPSs, GenClu's test generation can be multiple orders of magnitude faster than the prior state of the art. Further, when assessed via mutation testing, tests generated by GenClu were as good or better than anything else tested here. Hence, we recommend semi-supervised methods over prior methods (evolutionary search and fully-supervised learning). To enable open science, all the data and scripts used in this study are available at https://github.com/ai-se/CPS test generation.

show abstract

Research and Improvements on Mutation Operators for Simulink Models

Cited by 4 publications

References 7 publications

Test Generation and Test Prioritization for Simulink Models with Dynamic Behavior

Test Generation and Test Prioritization for Simulink Models with Dynamic Behavior

Automated test suite generation for time-continuous simulink models

What Not to Test (For Cyber-Physical Systems)

Contact Info

Product

Resources

About