Combining test case generation and runtime verification

Abstract-In case of underspecified or not fully predictable systems, models specifying system behaviors are nondeterministic. Nondeterminism poses several challenges for the validation and verification activities, including the problem of inconclusive tests in model-based testing with model checker. It is a validation technique that use model checker counterexamples as test cases.In this paper, we tackle the problem of testing nondeterministic systems by combining model-based testing and runtime conformance monitoring: the input sequences of the tests are automatically generated from nondeterministic models; then their execution is runtime monitored to check conformance of the code w.r.t. its specification. This technique provides an oracle for the test data, it never bears inconclusive responses, and it allows measuring the requirement coverage.The approach uses the Abstract State Machines as formal method for specification purposes and Java as implementation language. As a proof of concepts, the Tic-Tac-Toe game is taken as example of a system with nondeterministic behavior (both at specification and code levels).

show abstract

“…If more than one guard is true, Spin nondeterministically chooses only one corresponding rule to be executed 3 .…”

Section: Dealing With Nondeterminism a Test Generation In The Pmentioning

confidence: 99%

“…A way to combine model-based testing and runtime verification is presented in [3]. A test case generator, starting from a model of the input domain given as a nondeterministic Java program, produces inputs for the application using the Java PathFinder model checker that has been extended in order to perform symbolic execution.…”

Section: Related Workmentioning

confidence: 99%

Combining Model-Based Testing and Runtime Monitoring for Program Testing in the Presence of Nondeterminism

Arcaini

Gargantini

Riccobene

2013

2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops

View full text Add to dashboard Cite

show abstract

“…Other related hybrid techniques include the use of concrete execution to effectively "set-up" the environment for symbolic execution [50] and a combination of test case generation based on symbolic execution and runtime monitoring [6]; both these techniques have been applied in the context of NASA software systems. Furthermore, related approaches [34,70] seek to combine abstraction techniques, with automatic abstraction refinement, and theorem proving for program analysis and testing.…”

Section: Other Combined Analysesmentioning

confidence: 99%

A survey of new trends in symbolic execution for software testing and analysis

Păsăreanu

Visser

2009

Int J Softw Tools Technol Transfer

Self Cite

183

View full text Add to dashboard Cite

Symbolic execution is a well-known program analysis technique which represents program inputs with symbolic values instead of concrete, initialized, data and executes the program by manipulating program expressions involving the symbolic values. Symbolic execution has been proposed over three decades ago but recently it has found renewed interest in the research community, due in part to the progress in decision procedures, availability of powerful computers and new algorithmic developments. We provide here a survey of some of the new research trends in symbolic execution, with particular emphasis on applications to test generation and program analysis. We first describe an approach that handles complex programming constructs such as input recursive data structures, arrays, as well as multithreading. Furthermore, we describe recent hybrid techniques that combine concrete and symbolic execution to overcome some of the inherent limitations of symbolic execution, such as handling native code or availability of decision procedures for the application domain. We follow with a discussion of techniques that can be used to limit the (possibly infinite) number of symbolic configurations that need to be analyzed for the symbolic execution of looping programs. Finally, we give a short survey of interesting new applications, such as predictive testing, invariant inference,

show abstract

“…In this section we describe Eagle [2,3,1], the temporal logic framework for runtime verification used in this work. Most temporal logics were developed for use with model checkers, and so had to trade expressiveness for efficient model checking.…”

Section: Eaglementioning

confidence: 99%

Runtime verification for autonomous spacecraft software

Goldberg

Havelund

McGann

2005

2005 IEEE Aerospace Conference

View full text Add to dashboard Cite

-Autonomous systems are systems that can operate without human interference for extended periods of time in changing environments, likely in remote locations. Software is usually an essential part of such systems. However, adaptation of autonomy software is limited by its complexity and the difficulty of verifying and validating it. We describe an approach named runtime verification for testing autonomy software. Runtime verification is a technique for generating test oracles from abstract specifications of expected behavior. We describe its application to the PLASMA planning system, used in the recent Mars Exploration Rover missions. We furthermore discuss alternative autonomy V&V approaches.

show abstract

Combining test case generation and runtime verification

Cited by 75 publications

References 39 publications

Combining Model-Based Testing and Runtime Monitoring for Program Testing in the Presence of Nondeterminism

Combining Model-Based Testing and Runtime Monitoring for Program Testing in the Presence of Nondeterminism

A survey of new trends in symbolic execution for software testing and analysis

Runtime verification for autonomous spacecraft software

Contact Info

Product

Resources

About