Concrete hyperheuristic framework for test case prioritization

Abstract

Test case prioritization (TCP), which aims to find the optimal test case execution sequences for specific testing objects, has been widely used in regression testing. A wide variety of search methodologies and algorithms have been proposed to optimize test case execution sequences, namely, search‐based TCP. However, different algorithms perform differently and have different implementation costs and specific situations where an algorithm usually performs with high effectiveness and efficiency. When facing a new testing scenario, it is actually difficult to decide which algorithm is suitable. In this paper, to address the algorithm selection problem for different test scenarios, a more generally applicable algorithm based on a hyperheuristic strategy is proposed for search‐based TCP. This includes a range of multiobjective algorithms with a variety of crossover strategies and a learning agent strategy to evaluate and select the appropriate algorithm execution sequence dynamically for different scenarios. The concrete hyperheuristic framework for multiobjective TCP is presented with an algorithm's repository in the low level and the learning agent strategy in the higher level. Experiments show that the proposed learning agent strategy can accurately evaluate algorithms in multiobjective problems and select the appropriate algorithm in each iteration.

Regression Test Case Prioritization Frameworks: Challenges and Future Directions

“…This study reported an average APSC of 96.34% for statement coverage and 738.87 as an average of the EET when 100% statement coverage was achieved for several iterations compared to 18734.4 seconds of the average execution time with reduction of around 96% of the actual time. This study argues that the execution time for this framework is always shorter than the multi-objective PSO algorithm [7].…”

“…Bian, Li, Gao and Zhao [7] proposed a concrete hyper-heuristic framework for prioritizing test cases. The main purpose of this framework is to select, among several algorithms in the repository, the appropriate algorithm for different TCP scenarios.…”

“…Furthermore, F5 was evaluated using the APFD C metric, which is derived from the APFD, but incorporates the aspects of cost and fault severity and is calculated as follows [1], [25 Where t j is the cost of test case, and f i fault severity. Next, F9 was evaluated using a weighted average percentage of statement coverage (APSC), which is also derived from the APFD for scaling statement coverage and is calculated using the following equation [7], [33]:…”

“…A hybrid method combines TCP with other selection methods. In comparison to the selection and hybrid methods, test prioritization is considered as the most effective method in regression testing due to the higher possibility of finding hidden errors, as no test case is removed [7], [8].…”

Functional Requirement-Based Test Case Prioritization in Regression Testing: A Systematic Literature Review

“…TCP technique is proposed to define the execution order of test cases in a test suite according to given criteria. In other words, TCP is aimed to find an optimal text case execution to achieve specific testing objectives [116]. Maximum faults can be revealed from the execution of reordered test cases.…”