Scenario-Based Statistical Testing of Quality of Service Requirements

Beyer, Matthias; Dulz, Winfried

doi:10.1007/11495628_9

Cited by 7 publications

(7 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that the popular research branch of statistical testing, e.g.,[BD05,WRT00], is concerned with choosing the test inputs probabilistically; it does not test for the correctness of the random choices made by the system itself.…”

mentioning

confidence: 99%

Model-Based Testing of Probabilistic Systems

Gerhold

Stoelinga

2016

Fundamental Approaches to Software Engineering

View full text Add to dashboard Cite

This work presents an executable model-based testing framework for probabilistic systems with nondeterminism. We provide algorithms to automatically generate, execute and evaluate test cases from a probabilistic requirements specification. The framework connects input/output conformance-theory with hypothesis testing: our algorithms handle functional correctness, while statistical methods assess, if the frequencies observed during the test process correspond to the probabilities specified in the requirements. At the core of our work lies the conformance relation for probabilistic input/output conformance, enabling us to pin down exactly when an implementation should pass a test case. We establish the correctness of our framework alongside this relation as soundness and completeness; Soundness states that a correct implementation indeed passes a test suite, while completeness states that the framework is powerful enough to discover each deviation from a specification up to arbitrary precision for a sufficiently large sample size. The underlying models are probabilistic automata that allow invisible internal progress. We incorporate divergent systems into our framework by phrasing four rules that each well-formed system needs to adhere to. This enables us to treat divergence as the absence of output, or quiescence, which is a well-studied formalism in model-based testing. Lastly, we illustrate the application of our framework on three case studies.

show abstract

mentioning

confidence: 99%

Model-Based Testing of Probabilistic Systems

Gerhold

Stoelinga

2016

Fundamental Approaches to Software Engineering

View full text Add to dashboard Cite

show abstract

“…Looking ahead, different extensions and improvements are planed, such as calculating performance metrics [1] or determining the real-time capability for a given system specification. Last but not least, extending the TestPlayer tool is very easy due to the implementation in R and the integration based on the Eclipse plug-in StatET.…”

Section: Final Remarksmentioning

confidence: 99%

A comfortable testplayer for analyzing statistical usage testing strategies

Dulz

2011

Proceedings of the 6th International Workshop on Automation of Software Test

Self Cite

View full text Add to dashboard Cite

We will first give a brief introduction to a versatile modeling environment that allows the early validation of system specifications by a test-driven agile simulation approach. The main focus of our paper is on providing techniques for automated test case generation relying on statistical usage models. Based on the open source software R for statistical computing and graphics the easy to handle test case generation and analyzing TestPlayer c tool was developped. Starting from customer-specific graphical Markov chain usage models test cases are automatically generated and visualized by highlighting selected nodes and arcs in the model. In addition, various metrics and corresponding diagrams offer analytical techniques to assess the quality of the derived test suite.

show abstract

“…However, the existing methods for test case generation based on MCUMs lack the systematic integration of usage time in the MCUMs themselves, since they are discrete time Markov chains and each state transition corresponds to one time step on an abstract time base. Until now concepts are known to integrate timing aspects provisionally (Beyer and Dulz, 2005), but not coherently and in a manner that the usage time is integrated by keeping the semantics of the MCUM. Therefore, we present in this paper an extension of MCUMs that keeps the semantic and that supports automated test case generation for embedded systems in test environments as they are established in the automotive industry.…”

Section: Challenges In Automated Test Case Generationmentioning

confidence: 99%

“…These systems pose new challenges to testing. The approach presented in this paper is based on model based testing (Dalal et al, 1999), and can be further categorized into usage model based black box testing (Beyer and Dulz, 2005). Nowadays the focus in usage oriented testing based on Markov chain usage models (MCUM) lies in testing various sampled test cases and in testing that comes close to a known or estimated usage profile (Prowell, 2004).…”

Section: Introductionmentioning

confidence: 99%

MODEL DRIVEN TESTING WITH TIME AUGMENTED MARKOV CHAIN USAGE MODELS - Computations and Test Case Generation Algorithms for Time Augmented Markov Chain Usage Models

Siegl

Éntin²,

German

et al. 2009

Proceedings of the 4th International Conference on Software and Data Technologies

View full text Add to dashboard Cite

In statistical testing Markov Chain Usage Models (MCUM) are employed to describe the possible usage of the System-under-test (SUT) and to automatically derive test cases. However, MCUMs, as they are applied today, do not provide a universal concept for the integration of time. The estimated or known delay time of usage states and execution time of actions can only be considered by auxiliary means. But the importance of this aspect grows continously with the integration of increasingly complex and time sensitive systems in the automotive environment, such as hybrid vehicles. To overcome this drawback we present an intuitive extension of MCUMs that allows the consistent integration of time in MCUMs. We discuss how computations for MCUMs can be easily adapted to Time Augmented MCUMs and present algorithms for automatic test case generation from Time Augmented MCUMs together with a case study that has been conducted in the automotive domain.

show abstract

Scenario-Based Statistical Testing of Quality of Service Requirements

Cited by 7 publications

References 10 publications

Model-Based Testing of Probabilistic Systems

Model-Based Testing of Probabilistic Systems

A comfortable testplayer for analyzing statistical usage testing strategies

MODEL DRIVEN TESTING WITH TIME AUGMENTED MARKOV CHAIN USAGE MODELS - Computations and Test Case Generation Algorithms for Time Augmented Markov Chain Usage Models

Contact Info

Product

Resources

About