Adaptive Interaction Fault Location Based on Combinatorial Testing

Abstract

Combinatorial testing aims to detect interaction faults, which are triggered by interaction among parameters in system, by covering some specific combinations of parametric values. Most works about combinatorial testing focus on detecting such interaction faults rather than locating them. Based on the model of interaction fault schema, in which the interaction fault is described as a minimum fault schema and several corresponding parent-schemas, we propose an iterative adaptive interaction fault location technique for combinatorial testing. In order to locate interaction faults that detected in combinatorial testing, such technique utilizes delta debugging strategy to filtrate suspicious schemas by generating and running additional test cases iteratively. The properties, which include both recall and precision, of adaptive interaction fault location techniques are also analyzed in this paper. Analytical results suggest that the high scores in both recall and precision are guaranteed. It means that such technique can provide an efficient guidance for the applications of combinatorial testing.

BEN: A combinatorial testing-based fault localization tool

“…Currently, BEN only implements our own approach to fault localization. We plan to define and make public an API that allows BEN to be used by other combinatorial testingbased approaches, e.g., Inter-AIFL [ 12] and FIC_BS [ 14].…”

“…The first phase produces a ranking of combinations in terms of their likelihood to be failureinducing. A combination is failure-inducing, or simply inducing, if all tests containing this combination fail [ 3,5,7,12,13]. In the second phase, BEN takes a top ranked inducing combination from which a failed test and a small number of passed tests are generated.…”

“…Several approaches are reported on how to identify inducing combinations in a combinatorial test set [ 5,8,12,14]. Ma et al reported an approach that identifies faulty code based on failure-inducing combinations [ 6].…”

Recent Advances in Automatic Black-Box Testing

“…More formally, given a combinatorial strength t and a SUT with n parameters P ¼ {p 1 ,p 2 ,…,p n } with V i ¼ fv 1 , v 2 , …, v n i g denoting the set of valid values for each parameter p i , a CIT technique produces a t-way Covering Array (CA) defined as follows [144]: A CA is a two-dimensional array of size m  n, where the i-th column denotes the parameter p i and all the elements of this column are from the set V i , such that every subarray of size m  t contains each possible combinations of the corresponding t parameters values at least once. A t-way combinatorial test suite, covering all the parameters interaction of size t, can be easily obtained from a t-way CA by mapping each row of the covering array to a test case of the combinatorial test suite.…”

Heterogeneous Systems Testing Techniques: An Exploratory Survey

“…Wang et al [42] study the problem of how interaction faults can be located based on combinatorial testing rather than manual detection and propose a technique for interactive adaptive fault location. Results from this study show that the proposed technique performs better than the existing adaptive fault location techniques.…”