Incorporating Historical Test Case Performance Data and Resource Constraints into Test Case Prioritization

Fazlalizadeh, Yalda; Khalilian, Alireza; Azgomi, Mohammad Abdollahi; Parsa, Saeed

doi:10.1007/978-3-642-02949-3_5

Cited by 16 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Heuristic-based Test Prioritization. The proposed methods of this group have typically used heuristics based on information such as code coverage [3], [35], models [36], [37], history [4], [38], [39], and requirements [40], [41]. The main drawback of these methods, especially in a CI context, is that they are not adaptive to quickly changing environments.…”

Section: Related Workmentioning

confidence: 99%

Reinforcement Learning for Test Case Prioritization

Bagherzadeh¹,

Kahani²,

Briand³

2020

Preprint

View full text Add to dashboard Cite

Continuous Integration (CI) significantly reduces integration problems, speeds up development time, and shortens release time. However, it also introduces new challenges for quality assurance activities, including regression testing, which is the focus of this work. Though various approaches for test case prioritization have shown to be very promising in the context of regression testing, specific techniques must be designed to deal with the dynamic nature and timing constraints of CI. Recently, Reinforcement Learning (RL) has shown great potential in various challenging scenarios that require continuous adaptation, such as game playing, real-time ads bidding, and recommender systems. Inspired by this line of work and building on initial efforts in supporting test case prioritization with RL techniques, we perform here a comprehensive investigation of RL-based test case prioritization in a CI context. To this end, taking test case prioritization as a ranking problem, we model the sequential interactions between the CI environment and a test case prioritization agent as an RL problem, using three alternative ranking models. We then rely on carefully selected and tailored, state-of-the-art RL techniques to automatically and continuously learn an optimal test case prioritization strategy. Our extensive experimental analysis shows that the best RL solutions provide a significant accuracy improvement over previous RL-based work, with prioritization strategies getting close to being optimal, thus paving the way for using RL to prioritize test cases in a CI context.

show abstract

Section: Related Workmentioning

confidence: 99%

Reinforcement Learning for Test Case Prioritization

Bagherzadeh¹,

Kahani²,

Briand³

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Evaluation in the constraint environment is done differently than the nonconstrained one. This work has been modified by Fazlalizadeh et al [15] for quicker fault detection in constraint environment by proposing new equations considering effectiveness of test case, the history of execution, and the assigned least priority to them. The result was matched with random ordering and for visualizing the empirical result, box-plots were used for showing stability and quicker fault detection.…”

Section: Related Workmentioning

confidence: 99%

Efficiency Enhancement in Regression Test Case Prioritization Technique

Nayak,

Kumar,

Tripathi

et al. 2019

IJITEE

View full text Add to dashboard Cite

Regression testing is an important, but expensive, process that has a powerful impact on software quality. Unfortunately all the test cases, existing and newly added, cannot be re-executed due to insufficient resources. In this scenario, prioritization of test case helps in improving the efficacy of regression testing by arranging the test cases in such a way that the most beneficial (that has the potential to detect the more number of faults) are executed first. Previous work and existing prioritization techniques, though detects faults, but there is a need of improved techniques to enhance the process of regression testing by improving the fault detection rate. The new technique, proposed in this paper, gives improved result than the existing ones. The comparison of the effectiveness of the proposed approach is done with other prioritization and non-prioritization orderings. The result of the proposed approach shows higher average percentage of faults detected (APFD) values. Also, the performance is evaluated and it is observed that the capability of the proposed method outperforms other algorithms by enhancing the fault detection rate.

show abstract

“…proposed an approach on history based test case prioritization that uses the historical fault detection performance information of test cases for prioritization [20].…”

Section: Related Workmentioning

confidence: 99%