Evaluating and Comparing Fault-Based Testing Strategies for General Boolean Specifications: A Series of Experiments

Abstract

A great amount of fault-based testing strategies have been proposed to generate test cases for detecting certain types of faults in Boolean specifications. However, most of the previous studies on these strategies were focused on the Boolean expressions in the disjunctive normal form, even the irredundant disjunctive normal form -little work has been conducted to comprehensively investigate their performance on general Boolean specifications. In this study, we conducted a series of experiments to evaluate and compare eighteen fault-based testing strategies using over four thousand randomly generated fault-seeded Boolean expressions. In the experiments, a testing strategy is regarded as effective and efficient if it can detect most of the seeded faults using a small number of test cases. Our experimental results show that if a testing strategy is highly effective and efficient when testing the Boolean expressions in the irredundant disjunctive normal form, it also shows high effectiveness and efficiency on general Boolean expressions. It is found that one family of fault-based testing strategies, namely MUMCUT, normally deliver the best performance among all the eighteen strategies. Our study provides an in-depth understanding and insight of fault-based testing for general Boolean expressions.

Human Factors in Software Reliability Engineering

“…Fault-based approach has also been widely applied into the test case selection based on Boolean specifications, cf. [25].…”

CTFTP: A Test Case Generation Strategy for General Boolean Expressions Based on Ordered Binary Label-Driven Petri Nets

“…However, mutation patterns may be undetected for each type of fault in a general Boolean expression. Sun et al [10] evaluated and compared 18 fault-based testing strategies by means of a series of experiments, using more than 4000 randomly generated fault-based general Boolean expressions. Their experiments showed that the family of fault-based testing strategies, such as the MUMCUT, would usually provide the best performance.…”