Specification-based test oracles for reactive systems

Richardson, Debra J.; Aha, Stephanie Leif; O'Malley, T. Owen

doi:10.1145/143062.143100

Cited by 168 publications

(101 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We use the definition presented by Richardson et al that divides test oracles into two parts: the oracle information and the oracle procedure [11] . The oracle information specifies what is considered correct behavior, and the oracle procedure verifies that test executions correspond to the oracle information.…”

Section: Test Oraclesmentioning

confidence: 99%

“…Testing requires two key components: the test data and the test oracle. The test data are the inputs given to the application under test (AUT) and the test oracle is the artifact used to determine if the AUT executes correctly [11]. Clearly, both the test data and test oracle contribute to the effectiveness of the testing process; we cannot detect a fault if the test data does not drive execution to a state where the fault manifests itself, nor can we detect a fault if the test execution encounters the fault but the test oracle does not monitor any variables where the fault is revealed as a failure.…”

Section: Introductionmentioning

confidence: 99%

“…To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. effectiveness of test data selection criteria and methods of generating test data [3,7,9], considerably less work has been done on the effectiveness of test oracles and their impact on the testing process [4,16,11,12]. Consequently, while we know that the choice of test oracle influences the testing process, we have few, if any techniques for selecting the test oracle, and little understanding of how our selection impacts the testing process.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Better testing through oracle selection (NIER track)

Staats

Whalen

Heimdahl

2011

Proceedings of the 33rd International Conference on Software Engineering

View full text Add to dashboard Cite

In software testing, the test oracle determines if the application under test has performed an execution correctly. In current testing practice and research, significant effort and thought is placed on selecting test inputs, with the selection of test oracles largely neglected. Here, we argue that improvements to the testing process can be made by considering the problem of oracle selection. In particular, we argue that selecting the test oracle and test inputs together to complement one another may yield improvements testing effectiveness. We illustrate this using an example and present selected results from an ongoing study demonstrating the relationship between test suite selection, oracle selection, and fault finding.

show abstract

Section: Test Oraclesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Better testing through oracle selection (NIER track)

Staats

Whalen

Heimdahl

2011

Proceedings of the 33rd International Conference on Software Engineering

View full text Add to dashboard Cite

show abstract

“…Most of them appear as examples in literatures, such as elevator system [21], furnace system [22], alternating bit protocol [2], etc. The tool support described in the previous section is sufficient for many systems whose process behaviors either have no presence of data values or only have simple data values to enrich its behavioral patterns.…”

Section: Sensitivity Of (Compositional) Analysismentioning

confidence: 99%

Crafting a Promela Front-End with Abstract Data Types to Mitigate the Sensitivity of (Compositional) Analysis to Implementation Choices

Cheng

2005

Model Checking Software

View full text Add to dashboard Cite

Abstract.Recently, an active research topic in software verification is applying model checkers to programs, such as multi-threaded Java code. However, a program typically consists of more behaviors, such as operations on complicated data structures or implementation details which are typically made for some criteria like performance. A brute-force model extraction may produce a poor model for analysis engine. In this paper, we give examples to show how subtle changes in implementation may result in considerable differences in analysis, particularly to compositional analysis. Unfortunately, these implementation choices are made by programmers -people who typically do not possess the knowledge of verification. To mitigate such sensitivity, we advocate that verification tools should recognize and support abstract data types and, in the meantime, prohibit or suppress the use of array. Programming process behaviors with abstract data types can hide and converge the implementation choices. More importantly, abstract data types are informative. They provide essential information for tool automation to select a best implementation for analysis. In this paper, we describe the design and implementation of such a prototype tool which can parse systems written in Promela syntax.

show abstract

“…An important problem here is the gap between the system specification, which usually refers to high-level objects, and monitors, which refer to implementation-level events (such as function calls, etc.). Some issues related to derivation of monitors from system specification are considered in [17].…”

Section: Introductionmentioning

confidence: 99%