Prioritizing test cases for regression testing

Abstract

Test case prioritization techniques schedule test cases for execution in an order that attempts to increase their effectiveness at meeting some performance goal. Various goals are possible; one involves rate of fault detection -a measure of how quickly faults are detected within the testing process. An improved rate of fault detection during testing can provide faster feedback on the system under test and let software engineers begin correcting faults earlier than might otherwise be possible. One application of prioritization techniques involves regression testing -the retesting of software following modifications; in this context, prioritization techniques can take advantage of information gathered about the previous execution of test cases to obtain test case orderings. In this paper, we describe several techniques for using test execution information to prioritize test cases for regression testing, including: (1) techniques that order test cases based on their total coverage of code components; (2) techniques that order test cases based on their coverage of code components not previously covered; (3) techniques that order test cases based on their estimated ability to reveal faults in the code components that they cover. We report the results of several experiments in which we applied these techniques to various test suites for various programs and measured the rates of fault detection achieved by the prioritized test suites, comparing those rates to the rates achieved by untreated, randomly ordered, and optimally ordered suites. Analysis of the data shows that each of the prioritization techniques studied improved the rate of fault detection of test suites, and this improvement occurred even with the least expensive of those techniques. The data also shows, however, that considerable room remains for improvement. The studies highlight several cost-benefits tradeoffs among the techniques studied, as well as several opportunities for future work.

Improving Regression Testing Transparency and Efficiency with History-Based Prioritization -- An Industrial Case Study

“…Test prioritization can be used in conjunction with test case selection to set the execution order within the selected test suite. This will in turn ensure that if the session is unexpectedly terminated the time spent testing will have been more beneficial than if they were not prioritized [7].…”

“…The metrics used for comparing the techniques are accepted by the research community and have been used in several previous studies for the same purposes [22], [7], [13]. There is however other views on what is a good selection (e.g.…”

“…A metric introduced by Rothermel et al. [7] was used for this purpose: APFD, measuring the weighted average of the percentage of faults detected over the test session. A high value means that the total number of detected faults becomes high early in the session.…”

Kỹ Thuật Kiểm Thử Hồi Qui Hiệu Quả Cho Phát Triển Ứng Dụng Di Động

“…Có 2 loại lựa chọn ca kiểm thử theo thứ tự ưu tiên [7,21]: (1) Ưu tiên tổng quát là phương pháp lựa chọn và sắp xếp thứ tự ca kiểm thử có ảnh hưởng mức trung bình đối với các phiên bản phần mềm tiếp theo, (2) Specific Priorization sẽ liên quan đến việc chọn phiên bản cụ thể nào đó của chương trình. Theo Rothermel và cộng sự [21], R. Beena và S. Sarala [7], vấn đề xác định ưu tiên của ca kiểm thử được phát biểu như sau:…”

Adaptive Regression Testing Strategy: An Empirical Study

“…Test case prioritization techniques (e.g., [12], [6]) reorders test cases in order to increase the chance of early fault detection using various types of information available from software artifacts, such as the coverage of code achieved by tests, code change information, or code complexity. For example, one technique, total block coverage prioritization, simply sorts the test cases in the order of the number of blocks they cover.…”