Simulink fault localization: an iterative statistical debugging approach

Abstract

Debugging Simulink models presents a significant challenge in the embedded industry. In this work, we propose SimFL, a fault localization approach for Simulink models by combining statistical debugging and dynamic model slicing. Simulink models, being visual and hierarchical, have multiple outputs at different hierarchy levels. Given a set of outputs to observe for localizing faults, we generate test execution slices, for each test case and output, of the Simulink model. In order to further improve fault localization accuracy, we propose iSimFL, an iterative fault localization algorithm. At each iteration, iSimFL increases the set of observable outputs by including outputs at lower hierarchy levels, thus increasing the test oracle cost but offsetting it with significantly more precise fault localization. We utilize a heuristic stopping criterion to avoid unnecessary test oracle extension. We evaluate our work on three industrial Simulink models from Delphi Automotive. Our results show that, on average, SimFL ranks faulty blocks in the top 8.9% in the list of suspicious blocks. Further, we show that iSimFL significantly improves this percentage down to 4.4% by requiring engineers to observe only an average of five additional outputs at lower hierarchy levels on top of high-level model outputs.

Improving fault localization for Simulink models using search-based testing and prediction models

“…Formally, a Simulink model is a tuple (Nodes, Links, Inputs, Outputs) where Nodes is a set of Simulink blocks, Links ⊆ (Nodes × Nodes) is a set of links between the blocks, Inputs is a set of input ports and Outputs is a set of output ports. In our previous work [13], we have shown how statistical debugging can be extended and adapted to Simulink models. Statistical debugging utilizes an abstraction of program behavior, also known as spectra, (e.g., sequences of executed statements) obtained from testing.…”

“…We refer to each Simulink model spectrum as a test execution slice. A test execution slice is a set of (atomic) blocks that were executed by a test case to generate each output [13].…”

“…We denote the set of all test execution slices obtained by a test suite TS by TES TS . In [13], we provided a detailed discussion on how the static backward slices (i.e., static slice(o)) and test execution slices (i.e., tes tc,o ) can be computed for Simulink models.…”

“…In our previous work [13], we extended statistical debugging to Simulink models and evaluated the effectiveness of statistical debugging to localize faults in Simulink models. Our approach builds on a combination of statistical debugging and dynamic slicing of Simulink models.…”

“…Our approach builds on a combination of statistical debugging and dynamic slicing of Simulink models. We showed that the accuracy of our approach, when applied to Simulink models from the automotive industry, is comparable to the accuracy of statistical debugging applied to source code [13]. We further extended our approach to handle fault localization for Simulink models with multiple faults [14].…”