Constructing test cases for n-wise testing from tree-based test models

Abstract

In our previous work [17], we proposed a model-based combinatorial testing method, called FOT. It provides a technique to design test-models for combinatorial testing based on extended logic trees. In this paper, we introduce pair-wise testing (and by extension, n-wise testing, where n = 1,2,.. ) to FOT, by developing a technique to construct a testsuite of n-wise strategies from the test models in FOT. We take a "transformation approach" to realize this technique. To construct test suites, this approach first transforms testmodels in FOT, represented as extended logic trees, to those in the formats which the existing n-wise testing tools (such as PICT [9], ACTS [30], CIT-BACH [31], etc.) accept to input, and then applies transformed test-models to any of these tools. In this transformation approach, an algorithm, called "flattening algorithm", plays a key role. We prove the correctness of the algorithm, and implement the algorithm to automate such test-suite constructions, providing a tool called FOT-nw (FOT with n-wise). Further, to show the effectiveness of the technique, we conduct a case study, where we apply FOT-nw to design test models and automatically construct test suites of n-wise strategies for an embedded system of stationary services for real-use in industry.

Design of Prioritized N-Wise Testing

“…We are also considering to develop approximate algorithms with a lower computing cost, in case our technique is not scalable for practical sized SUT models. Future work also includes handling weights attached to both of parameters and values, and to structured SUT models [4].…”

Priority Integration for Weighted Combinatorial Testing

“…We prepared 60 weighted SUT models; 35 of them are collected from an existing benchmark set [3], 18 are from an industrial case study [5], and the other seven are from related work [1]. From these benchmarks, we removed the constraints and assigned weights using random numbers following a normal distribution with µ = 5.0 and σ = 1.0.…”