Abstract:This paper introduces a messy-GA for transition coverage of Simulink/StateFlow models. We introduce a tool that implements our approach and evaluate it on three benchmark embedded system Simulink models. Our messy-GA is able to achieve statistically significantly better coverage when compared to both random search and to a commercial tool for Simulink/StateFlow model Testing.
“…Here is a long (yet partial) list of just some of the testing problems with citations to a few example papers (of many) that adopt an SBST approach to find suitable test data: functional testing [118], safety testing [11], [32], security testing [41], robustness testing [104], integration testing [18], [26], service-based testing [24], temporal testing [19], [113], [119], exception testing [114], Combinatorial Interaction Testing (CIT) [20], [25], [95], (and Software Product Line (SPL) testing [48]), state [77] and state-based-model testing [30], [78] (including popular modelling notations such as MATLAB Simulink [90], [129]), and mutation based test [37], [49] and mutant [65], [92] generation. The State of the Art: SBST has made many achievements, and demonstrated its wide applicability and increasing uptake.…”
Abstract-Search Based Software Testing (SBST) formulates testing as an optimisation problem, which can be attacked using computational search techniques from the field of Search Based Software Engineering (SBSE). We present an analysis of the SBST research agenda 1 , focusing on the open problems and challenges of testing non-functional properties, in particular a topic we call 'Search Based Energy Testing' (SBET), Multi-objective SBST and SBST for Test Strategy Identification. We conclude with a vision of FIFIVERIFY tools, which would automatically find faults, fix them and verify the fixes. We explain why we think such FIFIVERIFY tools constitute an exciting challenge for the SBSE community that already could be within its reach.
“…Here is a long (yet partial) list of just some of the testing problems with citations to a few example papers (of many) that adopt an SBST approach to find suitable test data: functional testing [118], safety testing [11], [32], security testing [41], robustness testing [104], integration testing [18], [26], service-based testing [24], temporal testing [19], [113], [119], exception testing [114], Combinatorial Interaction Testing (CIT) [20], [25], [95], (and Software Product Line (SPL) testing [48]), state [77] and state-based-model testing [30], [78] (including popular modelling notations such as MATLAB Simulink [90], [129]), and mutation based test [37], [49] and mutant [65], [92] generation. The State of the Art: SBST has made many achievements, and demonstrated its wide applicability and increasing uptake.…”
Abstract-Search Based Software Testing (SBST) formulates testing as an optimisation problem, which can be attacked using computational search techniques from the field of Search Based Software Engineering (SBSE). We present an analysis of the SBST research agenda 1 , focusing on the open problems and challenges of testing non-functional properties, in particular a topic we call 'Search Based Energy Testing' (SBET), Multi-objective SBST and SBST for Test Strategy Identification. We conclude with a vision of FIFIVERIFY tools, which would automatically find faults, fix them and verify the fixes. We explain why we think such FIFIVERIFY tools constitute an exciting challenge for the SBSE community that already could be within its reach.
“…In [30] a messy genetic algorithm (GA) is used to generate transition tours through Simulink Stateflow models. The authors identify two main challenges: trigger blocks containing timing constraints or counters and cyclic paths which may require several traversals before triggering a transition.…”
Context: The generation of dynamic test sequences from a formal specification, complementing traditional testing methods in order to find errors in the source code. Objective: In this paper we extend one specific combinatorial test approach, the Classification Tree Method (CTM), with transition information to generate test sequences. Although we use CTM, this extension is also possible for any combinatorial testing method. Method: The generation of minimal test sequences that fulfill the demanded coverage criteria is an NP-hard problem. Therefore, search-based approaches are required to find such (near) optimal test sequences. Results: The experimental analysis compares the search-based technique with a greedy algorithm on a set of 12 hierarchical concurrent models of programs extracted from the literature. Our proposed search-based approaches (GTSG and ACOts) are able to generate test sequences by finding the shortest valid path to achieve full class (state) and transition coverage. Conclusion: The extended classification tree is useful for generating of test sequences. Moreover, the experimental analysis reveals that our search-based approaches are better than the greedy deterministic approach, especially in the most complex instances. All presented algorithms are actually integrated into a professional tool for functional testing.
“…This method focuses on the coverage of Stateflow model elements. Five articles [34,36,40,42,43] discuss test data generation. For example, Zhan and Clark [34] present a simulation-based testing framework for automatic test data generation.…”
Matlab Simulink is one of the major modeling and simulation tools applied in different embedded systems domain. Quality assurance is an essential, but often highly effort-consuming part of software development. A lot of different quality assurance techniques exist to ensure high quality, but these analysis and testing techniques are often applied in isolation. Therefore, we are interested in synergy effects when applying them in combination. Consequently, we performed a systematic mapping study to identify the current state of the art regarding such quality assurance techniques and existing combinations. Our main result is a classification of existing quality assurance techniques applied on Matlab Simulink models, and an overview of existing tool support and the validity of the approaches.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.