Locating Minimal Fault Interaction in Combinatorial Testing

Abstract

Combinatorial testing (CT) technique could significantly reduce testing cost and increase software system quality. By using the test suite generated by CT as input to conduct black-box testing towards a system, we are able to detect interactions that trigger the system's faults. Given a test case, there may be only part of all its parameters relevant to the defects in system and the interaction constructed by those partial parameters is key factor of triggering fault. If we can locate those parameters accurately, this will facilitate the software diagnosing and testing process. This paper proposes a novel algorithm named complete Fault Interaction Location (comFIL) to locate those interactions that cause system's failures and meanwhile obtains the minimal set of target interactions in test suite produced by CT. By applying this method, testers can analyze and locate the factors relevant to defects of system more precisely, thus making the process of software testing and debugging easier and more efficient. The results of our empirical study indicate that comFIL performs better compared with known fault location techniques in combinatorial testing because of its improved effectiveness and precision.

A Theory of Pending Schemas in Combinatorial Testing

“…Zheng et al [55] proposed an approach called comFIL (short for complete Fault Interaction Location), which contains two main steps. The first step selects the interactions that only appear in failing test cases, and the second step repeats generating and executing additional test cases for each candidate faulty schema to isolate the set of candidate faulty schemas until the MFS is obtained.…”

“…Second, for some approaches, their limitations are relatively small, and are not restricted to the numbers or de-grees of MFSs. These approaches include static approaches proposed by Yilmaz et al [9], Zhang et al [18], and Wang et al [36], and adaptive approaches proposed by Wang et al [15], Martínez et al [6], [17], Zheng et al [55], and Niu et al [13]. However, most of these approaches suffer from large computing complexity, and hence cannot be applied on systems with a large number of parameters and large values.…”

Two Improving Approaches for Faulty Interaction Localization using Logistic Regression Analysis

“…Another approaches are first to extract all candidate schemas that may be failure-inducing and then generate additional test cases to verify that they are indeed failure-inducing schemas [4,5,19,[26][27][28][29][30]. The AIFL by Shi et al [26] attempts to find a single failure-inducing schema, and the InterAIFL extended by Wang et al [19] can find multiple failure-inducing schemas.…”

“…The AIFL by Shi et al [26] attempts to find a single failure-inducing schema, and the InterAIFL extended by Wang et al [19] can find multiple failure-inducing schemas. ComFIL, proposed by Zheng et al [27], can find multiple failure-inducing schemas by elimination and generates test cases from all candidate schemas to reduce the most candidates in a single test. In addition, Niu et al [28] attempt to optimize the design of additional test cases by constructing a tuple relationship tree to describe the relationship between each candidate schema.…”

Hybrid Particle Swarm and Ranked Firefly Metaheuristic Optimization-Based Software Test Case Minimization

“…A novel algorithm named complete Fault Interaction Location was introduced in (Wei Zheng et al, 2016) for minimizing the fault interaction set of test. The algorithm failed to cover more faults since it does not use any optimization approach.…”