Prioritizing test cases with string distances

Abstract

Test case prioritisation aims at finding an ordering which enhances a certain property of an ordered test suite. Traditional techniques rely on the availability of code or a specification of the program under test. We propose to use string distances on the text of test cases for their comparison and elaborate a prioritisation algorithm. Such a prioritisation does not require code or a specification and can be useful for initial testing and in cases when code is difficult to instrument. In this paper, we also report on experiments performed on the "Siemens Test Suite", where the proposed prioritisation technique was compared with random permutations and four classical string distance metrics were evaluated. The obtained results, confirmed by a statistical analysis, indicate that prioritisation based on string distances is more efficient in finding defects than random ordering of the test suite: the test suites prioritized using string distances are more efficient in detecting the strongest mutants, and, on average, have a better APFD than randomly ordered test suites. The results suggest that string distances can be used for prioritisation purposes, and Manhattan distance could be the best choice.

Frameworks for Performing on Cloud Automated Software Testing Using Swarm Intelligence Algorithm: Brief Survey

“…Often, these test cases require a lot of time and effort even with automated testing [24,25,26]. Because each test consumes execution time, the execution time required generally decreases when parallelization is used.…”

“…Instead it ranks all existing test cases thus prioritizes the test cases. While testing is performed testers executes the test cases with the higher priorities first as long as the testing budged supports [25]. Following techniques are discussed in [25,28] to implement TCP.…”

“…al proposed a modified a TCP technique, which they called topic coverage, provides an alternative concept to code coverage. The goal was to rank tests so that they cover more topics sooner [25,28].…”

Regression Testing based on Hamming Distance and Code Coverage

“…Now, assume that fault F 3 has arisen, as shown in figure 1b, while debugging and fixing faults of F 1 and F 2 . According to hamming distance based prioritization proposed in [6,12], one of the two paths would be selected randomly. If path A-C-D-H is chosen randomly then the fault F 3 goes unnoticed.…”

“…But, code coverage based techniques do not always detect more bugs at the earliest. To overcome this, other techniques such as hamming distance based prioritization [6,12] were studied, which on the contrary ignored 100% code coverage. In this paper, we thus extend hamming distance based prioritization with code coverage technique to achieve both the goals of regression testing.…”