An analysis of test data selection criteria using the RELAY model of fault detection

Richardson, Debra J.; Thompson, Margaret C.

doi:10.1109/32.232020

Cited by 72 publications

(45 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, there is a problem of coincidental correctness when executing faulty programs over test cases (Richardson and Thompson, 1993;Hierons, 2006). For example, an activated fault may affect the program state during execution, but the failure of the state may not be propagated to the output to show a visible failure.…”

Section: Black-box Featuresmentioning

confidence: 99%

PAT: A pattern classification approach to automatic reference oracles for the testing of mesh simplification programs

Chan

Cheung

et al. 2009

Journal of Systems and Software

View full text Add to dashboard Cite

Graphics applications often need to manipulate numerous graphical objects stored as polygonal models. Mesh simplification is an approach to vary the levels of visual details as appropriate, thereby improving on the overall performance of the applications. Different mesh simplification algorithms may cater for different needs, producing diversified types of simplified polygonal model as a result. Testing mesh simplification implementations is essential to assure the quality of the graphics applications. However, it is very difficult to determine the oracles (or expected outcomes) of mesh simplification for the verification of test results.A reference model is an implementation closely related to the program under test. Is it possible to use such reference models as pseudo-oracles for testing mesh simplification programs? If so, how effective are they?This paper presents a fault-based pattern classification methodology, called PAT, to address the questions. In PAT, we train the C4.5 classifier using black-box features of samples from a reference model and its fault-based versions, in order to test samples from the subject program. We evaluate PAT using four implementations of mesh simplification algorithms as reference models applied to 44 open-source three-dimensional polygonal models. Empirical results reveal that the use of a reference model as a pseudo-oracle is effective for testing the implementations of resembling mesh simplification algorithms. However, the results also show a tradeoff: When compared with a simple reference model, the use of a resembling but sophisticated reference model is more effective and accurate but less robust.

show abstract

Section: Black-box Featuresmentioning

confidence: 99%

PAT: A pattern classification approach to automatic reference oracles for the testing of mesh simplification programs

Chan

Cheung

et al. 2009

Journal of Systems and Software

View full text Add to dashboard Cite

show abstract

“…In the past, various types of faults have been defined and used to study effectiveness of test criteria; see for example [16,22,25,30,33]. An analysis of various fault classes and the hierarchy among them is presented in [15,27].…”

Section: Types Of Faultsmentioning

confidence: 99%

“…An approach, known as fault based testing [16,22], attempts to alleviate the problem of test-set reliability by seeking a solution through hypothesizing faults that can occur in the implementation. The reliability of a technique is measured in terms of these hypothesized faults.…”

Section: Related Workmentioning

confidence: 99%

Experimental evaluation of the tolerance for control‐flow test criteria

Kapoor

Bowen

2004

Software Testing Verif & Rel

View full text Add to dashboard Cite

For a given test criterion, the number of test-sets satisfying the criterion may be very large, with varying fault detection effectiveness. In recent work [29], the measure of variation in effectiveness of test criterion was defined as 'tolerance'. This paper presents an experimental evaluation of tolerance for control-flow test criteria. The experimental analysis is done by exhaustive test-set generation, wherever possible, for a given criteria which improves on earlier empirical studies that adopted analysis of some test-sets using random selection techniques. Four industrially used control-flow testing criteria, Condition Coverage (CC), Decision Condition Coverage (DCC), Full Predicate Coverage (FPC) and Modified Condition/Decision Coverage (MC/DC) have been analysed against four types of faults. A new test criteria, Reinforced Condition/Decision Coverage (RC/DC) [28], is also analysed and compared. The Boolean specifications considered were taken from a past research paper and also generated randomly. To ensure that it is the test-set property that influences the effectiveness and not the test-set size, the average test-set size was kept same for all the test criteria except RC/DC. A further analysis of variation in average effectiveness with respect to number of conditions in the decision was also undertaken. The empirical results show that the MC/DC criterion is more reliable and stable when compared to the other considered criteria. Though the number of test-cases is large in RC/DC testsets, no significant improvement in effectiveness and tolerance was observed over MC/DC.

show abstract

“…In this section, we adopt the RELAY model [10] and introduce origination condition and propagation condition [9] to define Boolean difference model to compute detection conditions for various fault classes.…”

Section: Detection Conditionsmentioning

confidence: 99%

“…[9] removed the restriction of IDNF and presented a more general analysis of literal faults w.r.t. negation, reference and insertion (conjunction or disjunction) using the RELAY model [10].…”

Section: Sac'08mentioning

confidence: 99%

A detectability analysis of fault classes for Boolean specifications

Chen

Nie

2008

Proceedings of the 2008 ACM Symposium on Applied Computing

View full text Add to dashboard Cite

The detectability hierarchies of fault classes for specificationbased testing have been established to prioritize test cases so as to achieve earlier detection of more faults. This paper extends and complements the existing studies by analyzing detectability of fault classes for Boolean specifications in general form. The monotonicity of detectability is discovered that a fault occurs at literal is more difficult to detect than the corresponding one occurs at other positions. Furthermore, a strong detectability relationship is introduced to overcome the flaw of traditional approach. It can help identify stronger faults and skip weaker faults during testing. As a result, two detectability hierarchies are established on the detection conditions of fault classes.

show abstract

An analysis of test data selection criteria using the RELAY model of fault detection

Abstract: To appear in Transactions on Software Engineering.

Cited by 72 publications

References 36 publications

PAT: A pattern classification approach to automatic reference oracles for the testing of mesh simplification programs

PAT: A pattern classification approach to automatic reference oracles for the testing of mesh simplification programs

Experimental evaluation of the tolerance for control‐flow test criteria

A detectability analysis of fault classes for Boolean specifications

Contact Info

Product

Resources

About