Covering array generation (CAG) is the key research problem in combinatorial testing and is an NP-complete problem. With the increasing complexity of software under test and the need for higher interaction covering strength t, the techniques for constructing highstrength covering arrays are expected. This paper presents a hybrid heuristic memetic algorithm named QSSMA for high-strength CAG problem. The sub-optimal solution acceptance rate is introduced to generate multiple test cases after each iteration to improve the efficiency of constructing high-covering strength test suites. The QSSMA method could successfully build high-strength test suites for some instances where t up to 15 within one day cutoff time and report five new best test suite size records. Extensive experiments demonstrate that QSSMA is a competitive method compared to state-of-theart methods.
K E Y W O R D S optimisation, particle swarm optimisation, software engineering
| INTRODUCTIONSoftware testing is an important area of software engineering and has great significance in ensuring software quality. Due to the large amount of test data required by modern software, if a complete test is performed, the size of the test data will increase sharply as increasing parameters or parameter values, leading to a lower testing efficiency. Thus, effective failure detection at a low cost is a critical issue for test case generation. Combinatorial testing (CT), a popular testing method, can detect failures triggered by various interactions of factors [1].There are two key issues in the field of combinatorial testing: (1) how to construct a test suite as small as possible; (2) how to find the combination of parameters that trigger the fault in a particular test case that triggers a software under test (SUT) failure. Researchers are more concerned with problem one, known as covering array generation (CAG) [2]. The main idea of CT is to construct a test suite represented by a covering array (CA) that contains all combinations of parameter's value of the SUT at least once and hopes the array is as small as possible. The generation of the smallest size test suite has been proven to be an NP-complete problem [3]. Numerous methods have been proposed, including mathematical, greedy, and artificial intelligence-based search methods. However, all the proposed mathematical methods apply only to restrictive parameter sets. Besides, greedy algorithms often result in oversized test suites. Due to this limitation, scholars focus more on AI-based (heuristic) search methods [4].Many efficient heuristic search algorithms have been successfully applied to CA generation, including geneticThis is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.