Flake It 'Till You Make It

Parry, Owain; Kapfhammer, Gregory M.; Hilton, Michael; McMinn, Phil

doi:10.1145/3387940.3392177

Cited by 8 publications

(1 citation statement)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this approach, testing process is deriving test inputs from the internal structure of the software under test [17]. This approach treating the software as group of structure such as branches, paths and statement.…”

Section: White (Structural) Box Testingmentioning

confidence: 99%

Software Testing Based on Research: A Road Map

Salim Abdulla,

Abdul Qader Thabit,

Ahmed Al-Zaidi

2023

ejeee

View full text Add to dashboard Cite

This paper serves as a guide for both researchers and students who are new to the research area of Search-Based Package Testing. The application of metaheuristic explore methods in this context specifically pertains to utilizing these algorithms for generating test data. In the realm of software engineering research, software testing emerges as a robust and fertile ground for exploration. The integration of AI methods into software program testing is an evolving research direction. Often, newcomers to this field face challenges due to limited knowledge about the interaction between software testing and artificial intelligence. This paper aims to provide a roadmap for these new researchers or students in the field.

show abstract

Section: White (Structural) Box Testingmentioning

confidence: 99%

Software Testing Based on Research: A Road Map

Salim Abdulla,

Abdul Qader Thabit,

Ahmed Al-Zaidi

2023

ejeee

View full text Add to dashboard Cite

show abstract

What We Talk About When We Talk About Software Test Flakiness

Barboni

Bertolino

Angelis

2021

Communications in Computer and Information Science

View full text Add to dashboard Cite

Empirically evaluating flaky test detection techniques combining test case rerunning and machine learning models

Parry¹,

Kapfhammer²,

Hilton³

et al. 2023

Empir Software Eng

View full text Add to dashboard Cite

A flaky test is a test case whose outcome changes without modification to the code of the test case or the program under test. These tests disrupt continuous integration, cause a loss of developer productivity, and limit the efficiency of testing. Many flaky test detection techniques are rerunning-based, meaning they require repeated test case executions at a considerable time cost, or are machine learning-based, and thus they are fast but offer only an approximate solution with variable detection performance. These two extremes leave developers with a stark choice. This paper introduces CANNIER, an approach for reducing the time cost of rerunning-based detection techniques by combining them with machine learning models. The empirical evaluation involving 89,668 test cases from 30 Python projects demonstrates that CANNIER can reduce the time cost of existing rerunning-based techniques by an order of magnitude while maintaining a detection performance that is significantly better than machine learning models alone. Furthermore, the comprehensive study extends existing work on machine learning-based detection and reveals a number of additional findings, including (1) the performance of machine learning models for detecting polluter test cases; (2) using the mean values of dynamic test case features from repeated measurements can slightly improve the detection performance of machine learning models; and (3) correlations between various test case features and the probability of the test case being flaky.

show abstract

Flake It 'Till You Make It

Cited by 8 publications

References 6 publications

Software Testing Based on Research: A Road Map

Software Testing Based on Research: A Road Map

What We Talk About When We Talk About Software Test Flakiness

Empirically evaluating flaky test detection techniques combining test case rerunning and machine learning models

Contact Info

Product

Resources

About