An empirical evaluation of a three‐tier conduit framework for multifaceted test case classification and selection using fuzzy‐ant colony optimisation approach

Abstract

Summary The test case optimisation is an NP‐complete, knowledge‐driven, data‐driven, and multidimensional search space partitioning and dimension reduction problem. In the multifaceted test case classification, partitioning and reducing the multidimensional test case fitness search space is the critical problem. The vague nature of fitness parameters, conflicting nature objectives, and ambiguity in the test case fitness evaluation have created and increased the uncertainty, the imprecision, and the incompleteness in the test case classification and selection. Because of the increasing ambiguity, the complexity, and the cost of software testing, automated test case classification and selection has emerged as an appropriate tool to classify test cases into predefined categories using the multifaceted concept. Most of the test cases affecting the performance of the classifier are irrelevant and redundant. A strong need therefore exists to devise an intelligent technique to identify and remove test cases affecting the performance of the classifier. For increasing the performance of classifier, multifaceted test case selection is used to reduce fitness search space to be searched. In this paper, a three‐tier sequential framework is proposed for a multifaceted test case classification and selection. The first stage of the proposed framework is the fuzzy synthesis‐based filtration approach for multifaceted test case fitness evaluation and classification. The second stage of the proposed framework is the fuzzy entropy‐based filtration technique with a backward search strategy, used for estimating and reducing the ambiguity in test case fitness evaluation, classification, and selection. The third stage of the proposed framework is the ant colony optimisation‐based wrapper technique with a forward search strategy, employed to select test cases from the output (reduced) test suite by the second stage. The proposed framework is tested on artefacts of benchmark applications. The results of the empirical study clearly show that the third stage of our proposed method outperforms the second and first stages, and the performance of the algorithms used in all three stages increases on average as the stages are escalating. The classification accuracy is enhanced by reducing the ambiguity in fitness and the classification of test cases, increasing the number of test cases accurately classified, and reducing the number in the test case pool to be exercised. Copyright © 2014 John Wiley & Sons, Ltd.

An Insight Into Test Case Optimization: Ideas and Trends With Future Perspectives

“…Hybrid of fuzzy entropy and ant colony optimization has been used by Kumar et al [20] for classification and selection of test cases. It consists of three stages.…”

“…• Fuzzy Entropy value of test cases: It's been proposed by Kumar et al [20] which measures ambiguousness of test cases. It can be applied in the fitness, classification, and selection of test cases to measure the ambiguity.…”

“…During literature various recently proposed adequacy criteria were identified such as behavioral adequacy [41], fuzzy entropy of test cases [20], maximizing minimum sum of coverage [15], diversity aware mutation adequacy criteria [42], and operational coverage [75] which has already been discussed in Section 4.5.…”

“…Ambiguity based classification of test cases for test case selection has been done only by Kumar et al [20]. They have used fuzzy entropy-based technique to select low ambiguity test cases and then tested the software under test.…”

PHOEBE: an automation framework for the effective usage of diagnosis tools in the performance testing of clustered systems

“…The idea of applying automation in the performance testing domain is not new. However, most of the research has focused on automating the generation of load test suites . For example, the authors in propose an approach to automate the generation of test cases based on specified levels of load and combinations of resources.…”