Model-based testing of stochastically timed systems

Gerhold, Marcus; Hartmanns, Arnd; Stoelinga, Mariëlle

doi:10.1007/s11334-019-00349-z

Cited by 5 publications

(5 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to this refinement process autonomously operated by the framework, at any time, domain experts can, in principle, make changes to the pool P, e.g., by manually specifying a new formula encoding a bad behavior. 3…”

Section: The General Frameworkmentioning

confidence: 99%

“…Let us now consider some examples of the bSTL formulas used within the framework. An example for the Backblaze dataset is formula (G [0,2] SMART 194 >45.6) ∧ (F [2,3] SMART 198 >0.32). Such a formula makes evident a bad behavior where the hard drive maintains a temperature exceeding 45.6 • C in the first 3 days, and then, in the following 2 days, its uncorrectable sector count becomes greater than 0.…”

Section: ) Online Evaluationmentioning

confidence: 99%

“…To overcome these limitations, various methods, that combine classic exhaustive formal verification techniques with model-based testing and monitoring, have recently been proposed in the literature (see, e.g., [2], [3]). Here, we focus on the latter.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Monitors That Learn From Failures: Pairing STL and Genetic Programming

et al. 2023

View full text Add to dashboard Cite

In several domains, systems generate continuous streams of data during their execution, including meaningful telemetry information, that can be used to perform tasks like preemptive failure detection. Deep learning models have been exploited for these tasks with increasing success, but they hardly provide guarantees over their execution, a problem which is exacerbated by their lack of interpretability. In many critical contexts, formal methods, which ensure the correct behavior of a system, are thus necessary. However, specifying in advance all the relevant properties and building a complete model of the system against which to check them is often out of reach in real-world scenarios. To overcome these limitations, we design a framework that resorts to monitoring, a lightweight runtime verification technique that does not require an explicit model specification, and pairs it with machine learning. Its goal is to automatically derive relevant properties, related to a bad behavior of the considered system, encoded by means of formulas of Signal Temporal Logic (STL). Results based on experiments performed on well-known benchmark datasets show that the proposed framework is able to effectively anticipate critical system behaviors in an online setting, providing human-interpretable results.INDEX TERMS Machine learning, formal methods, runtime verification, monitoring, failure detection, explainable AI.

show abstract

Section: The General Frameworkmentioning

confidence: 99%

Section: ) Online Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Monitors That Learn From Failures: Pairing STL and Genetic Programming

et al. 2023

View full text Add to dashboard Cite

show abstract

“…The authors in [10] suggest using multiple unique input/output (UIO) sequences to generate a test sequence and a method to minimize input/output operations as well as external coordination messages. Moreover, two model-based testing frameworks are explained in [11]. They additionally cover the stochastic aspects in hard and soft real-time systems.…”

Section: Literature Reviewmentioning

confidence: 99%

An Adaptive Approach to Optimize Probabilistic Distributed Testing

Tajioue¹,

Maakoul²,

Moutai³

et al. 2022

Stat., optim. inf. comput.

View full text Add to dashboard Cite

Typically, conformance testing consists of placing a set of parallel testers at each port of an implementation to ensure its conformance to the specifification. However, a number of common fault detections occur if no coordination is made between these parallel testers and the implementation under test (IUT). Therefore, the test process must support mechanisms of coordination between these distributed components, particularly for implementation with stochastic behaviour. To this end, as well as to analyse the stochastic behaviour of the implementation under test, we propose in this paper an algorithm to generate for each tester a probabilistic local test sequence (PLTS) aiming to avoid both synchronization and observation issues. Finally, we suggest a new architecture based on Markov decision processes with an adaptive controller to control and optimize the whole testing process.

show abstract

“…MA are the semantic basis for generalised stochastic Petri nets [37] and dynamic extensions of fault trees [12,61]. They have been applied to various problems including model-based testing [40] and robot planning under uncertainty [4,67]. Several publications studied algorithmic problems related to the efficient analysis of MA [3, 21, 23-25, 43, 44, 55].…”

Section: Introductionmentioning

confidence: 99%

A Modest Approach to Markov Automata

Butkova

Hartmanns

Hermanns

2021

ACM Trans. Model. Comput. Simul.

Self Cite

View full text Add to dashboard Cite

Markov automata are a compositional modelling formalism with continuous stochastic time, discrete probabilities, and nondeterministic choices. In this article, we present extensions to M ODEST , an expressive high-level language with roots in process algebra, that allow large Markov automata models to be specified in a succinct, modular way. We illustrate the advantages of M ODEST over alternative languages. Model checking Markov automata models requires dedicated algorithms for time-bounded and long-run average reward properties. We describe and evaluate the state-of-the-art algorithms implemented in the mcsta model checker of the M ODEST T OOLSET . We find that mcsta improves the performance and scalability of Markov automata model checking compared to earlier and alternative tools. We explain a partial-exploration approach based on the BRTDP method designed to mitigate the state space explosion problem of model checking, and experimentally evaluate its effectiveness. This problem can be avoided entirely by purely simulation-based techniques, but the nondeterminism in Markov automata hinders their straightforward application. We explain how lightweight scheduler sampling can make simulation possible, and provide a detailed evaluation of its usefulness on several benchmarks using the M ODEST T OOLSET ’s modes simulator.

show abstract

Model-based testing of stochastically timed systems

Cited by 5 publications

References 49 publications

Monitors That Learn From Failures: Pairing STL and Genetic Programming

Monitors That Learn From Failures: Pairing STL and Genetic Programming

An Adaptive Approach to Optimize Probabilistic Distributed Testing

A Modest Approach to Markov Automata

Contact Info

Product

Resources

About