Regression test selection across JVM boundaries

Çelik, Ahmet; Vasić, Marko; Milićević, Aleksandar; Gligoric, Milos

doi:10.1145/3106237.3106297

Cited by 32 publications

(59 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a number of changes along with a number of tests grow in many giant repositories, it may not be feasible to perform all tests during builds. To address this issue, several solutions have been proposed based on static analysis [17,33,40,46], dynamic analysis [23,39], and datadriven [32] using a ML model. Therefore, upon committing a change, and instead of conducting all test cases, only a subset of test cases are required to run.…”

Section: Related Workmentioning

confidence: 99%

Keeping Master Green at Scale

Ananthanarayanan¹,

Ardekani²,

Haenikel³

et al. 2019

Proceedings of the Fourteenth EuroSys Conference 2019

View full text Add to dashboard Cite

Giant monolithic source-code repositories are one of the fundamental pillars of the back end infrastructure in large and fast-paced software companies. The sheer volume of everyday code changes demands a reliable and efficient change management system with three uncompromisable key requirements-always green master, high throughput, and low commit turnaround time. Green refers to a master branch that always successfully compiles and passes all build steps, the opposite being red. A broken master (red) leads to delayed feature rollouts because a faulty code commit needs to be detected and rolled backed. Additionally, a red master has a cascading effect that hampers developer productivitydevelopers might face local test/build failures, or might end up working on a codebase that will eventually be rolled back. This paper presents the design and implementation of SubmitQueue. It guarantees an always green master branch at scale: all build steps (e.g., compilation, unit tests, UI tests) successfully execute for every commit point. SubmitQueue has been in production for over a year, and can scale to thousands of daily commits to giant monolithic repositories.

show abstract

Section: Related Workmentioning

confidence: 99%

Keeping Master Green at Scale

Ananthanarayanan¹,

Ardekani²,

Haenikel³

et al. 2019

Proceedings of the Fourteenth EuroSys Conference 2019

View full text Add to dashboard Cite

show abstract

“…Because of that, Zhang [15] proposed a hybrid test selection technique that combined file-level and method-level analysis. Furthermore, Celik et al [8] designed a file-level test selection across JVM boundaries, which can find more dependencies and improve precision of selection.…”

Section: Related Workmentioning

confidence: 99%

A Class-level Test Selection Approach Toward Full Coverage For Continuous Integration

Li¹,

Wang²,

Wang³

et al. 2019

International Conferences on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

Continuous Integration (CI) is an important practice in agile development. With the growth of integration system, running all tests to verify the quality of submitted code, is clearly uneconomical. This paper aims at selecting a proper test subset towards full coverage of all changed and affected code so as to reduce the cost of CI testing. We proposes FEST, a novel approach, which searches for the full dependencies of changed code at the class level and then selects test classes related to the changed and affected classes. We assess FEST from fault detection efficiency and cost effectiveness based on 18 open source projects with 261 continuous integration versions from Eclipse and Apache communities, and compare it with the stateof-the-art approach ClassSRTS (as baseline). Results show that FEST (1) can not only cover all faults detected by actual CI testing and baseline, but also find new faults in 25% and 18% versions respectively. (2) shows better or equal test scale benefits than actual CI testing (in 98% versions) and baseline (in 99% versions); and can compensate risk of omitting necessary tests for actual CI testing (in 62% versions) and baseline (in 73% versions).

show abstract

“…An approach has been proposed for regression test selection with dynamic test dependencies, where, for each test method or a test class file, dependencies are computed. Another approach by this author has been to dynamically detect, at the operating system level, all file artifacts a test depends on . Legunsen et al reported a study evaluating the performance of static regression test selection techniques, and the authors reported that class‐level approaches perform better than method‐level approaches.…”

Section: Related Workmentioning

confidence: 99%

“…Another approach by this author has been to dynamically detect, at the operating system level, all file artifacts a test depends on. 34 Legunsen et al reported a study 35 evaluating the performance of static regression test selection techniques, and the authors reported that class-level approaches perform better than method-level approaches. Zhang proposed a hybrid regression test selection approach combining method and file level selection.…”

Section: Related Workmentioning

confidence: 99%

A learning algorithm for optimizing continuous integration development and testing practice

2018

View full text Add to dashboard Cite

Summary Continuous integration, at its core, includes a set of practices that aim to prevent and reduce the cost of software integration issues by merging working software copies often. Regression testing is considered a good practice in software development with continuous integration, which ensures that code changes are not negatively affecting software functionality. As, nowadays, software development is carried out iteratively, with small code increments continuously developed and regression tested, it is of critical importance that continuous regression testing is time efficient. However, in practice, regression testing is often long lasting and faces scalability problems as software grows larger or as software changes are made more frequently. One contributing factor to these issues is test redundancy, which causes the same software functionality being tested multiple times across a test suite. In large‐scale software, especially highly configurable software, redundancy in continuous regression testing can significantly grow the size of test suites and negatively affect the cost effectiveness of continuous integration. This paper presents a practical learning algorithm for optimizing continuous integration testing by reducing ineffective test redundancy in regression suites. The novelty of the algorithm lies in learning and predicting the fault‐detection effectiveness of continuous integration tests using historical test records and combining this information with coverage‐based redundancy metrics. The goal is to identify ineffective redundancy, which is maximally reduced in the resulting regression test suite, thus reducing test time and improving the performance of continuous integration. We apply and evaluate the algorithm in two industrial projects of continuous integration. The results show that the proposed algorithm can improve the efficiency of continuous integration practice in terms of decreasing test execution time by 38% on average compared to the industry practice of our case study and by 40% on average compared to the retest‐all approach. The results further demonstrate no significant reduction in fault‐detection effectiveness of continuous regression testing. This suggests that the proposed algorithm contributes to the state of the practice in the continuous integration development and testing of highly configurable systems.

show abstract

Regression test selection across JVM boundaries

Cited by 32 publications

References 42 publications

Keeping Master Green at Scale

Keeping Master Green at Scale

A Class-level Test Selection Approach Toward Full Coverage For Continuous Integration

A learning algorithm for optimizing continuous integration development and testing practice

Contact Info

Product

Resources

About