An empirical study of the long duration of continuous integration builds

Ghaleb, Taher Ahmed; Costa, Daniel Alencar da; Zou, Ying

doi:10.1007/s10664-019-09695-9

Cited by 53 publications

(14 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A popular effort to reduce the cost of CI focuses on understanding what causes long build durations e.g., [17], [58]. Thus, most of the approaches that reduce the cost of CI aim at making builds faster by running fewer test cases on each build.…”

Section: Approaches To Reduce Cost O F CImentioning

confidence: 99%

What Helped, and what did not? An Evaluation of the Strategies to Improve Continuous Integration

Jin

Servant

2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Continuous integration (CI) is a widely used practice in modern software engineering. Unfortunately, it is also an expensive practice -Google and Mozilla estimate their CI systems in millions of dollars. There are a number of techniques and tools designed to or having the potential to save the cost of CI or expand its benefit -reducing time to feedback. However, their benefits in some dimensions may also result in drawbacks in others. They may also be beneficial in other scenarios where they are not designed to help. In this paper, we perform the first exhaustive comparison of techniques to improve CI, evaluating 14 variants of 10 techniques using selection and prioritization strategies on build and test granularity. We evaluate their strengths and weaknesses with 10 different cost and time-tofeedback saving metrics on 100 real-world projects. We analyze the results of all techniques to understand the design decisions that helped different dimensions of benefit. We also synthesized those results to lay out a series of recommendations for the development of future research techniques to advance this area.

show abstract

Section: Approaches To Reduce Cost O F CImentioning

confidence: 99%

What Helped, and what did not? An Evaluation of the Strategies to Improve Continuous Integration

Jin

Servant

2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

show abstract

“…The phenomenon of slow builds was also investigated by Ghaleb et al (2019), finding that long builds (e.g., exceeding the 10 minutes rule-of-thumb (Duvall et al, 2007)) do not necessarily depend on the project size/complexity, but may also be related to build configuration issues, such as multiple (and failed) build attempts. To this extent, our catalog includes various kinds of build configuration issues that can result in such a side effect.…”

Section: Related Workmentioning

confidence: 99%

An empirical characterization of bad practices in continuous integration

et al. 2020

View full text Add to dashboard Cite

Continuous Integration (CI) has been claimed to introduce several benefits in software development, including high software quality and reliability. However, recent work pointed out challenges, barriers and bad practices characterizing its adoption. This paper empirically investigates what are the bad practices experienced by developers applying CI. The investigation has been conducted by leveraging semi-structured interviews of 13 experts and mining more than 2,300 Stack Overflow posts. As a result, we compiled a catalog of 79 CI bad smells belonging to 7 categories related to different dimensions of a CI pipeline management and process. We have also investigated the perceived importance of the identified bad smells through a survey involving 26 professional developers, and discussed how the results of our study relate to existing knowledge about CI bad practices. Whilst some results, such as the poor usage of branches, confirm existing literature, the study also highlights uncovered bad practices, e.g., related to static analysis tools or the abuse of shell scripts, and contradict knowledge from existing literature, e.g., about avoiding nightly builds. We discuss the implications of our catalog of CI bad smells for (i) practitioners, e.g., favor specific, portable tools over hacking, and do not ignore nor hide build failures, (ii) educators, e.g., teach CI culture, not just technology, and teach CI by providing examples of what not to do, and (iii) researchers, e.g., developing support for failure analysis, as well as automated CI bad smell detectors.

show abstract

“…Besides general build failures, several studies focused on specific type of build failures. Seo et al [25] performed a large scale study on compiler errors in build process at Google; Zhang et al [32] further investigated compiler errors on open-source Java projects in continuous integration; Beller et al [9] investigated build breakages related to testing on TRAVIS CI; Ghaleb et al [12] studied the build issues with long duration.…”

Section: Build Failuresmentioning

confidence: 99%

Understanding build issue resolution in practice: symptoms and fix patterns

Lou

Chen

Cao

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

Build systems are essential for modern software maintenance and development, while build failures occur frequently across software systems, inducing non-negligible costs in development activities. Build failure resolution is a challenging problem and multiple studies have demonstrated that developers spend non-trivial time in resolving encountered build failures; to relieve manual efforts, automated resolution techniques are emerging recently, which are promising but still limitedly effective. Understanding how build failures are resolved in practice can provide guidelines for both developers and researchers on build issue resolution. Therefore, this work presents a comprehensive study of fix patterns in practical build failures. Specifically, we study 1,080 build issues of three popular build systems Maven, Ant, and Gradle from Stack Overflow, construct a fine-granularity taxonomy of 50 categories regarding to the failure symptoms, and summarize the fix patterns for different failure types. Our key findings reveal that build issues stretch over a wide spectrum of symptoms; 67.96% of the build issues are fixed by modifying the build script code related to plugins and dependencies; and there are 20 symptom categories, more than half of whose build issues can be fixed by specific patterns. Furthermore, we also address the challenges in applying non-intuitive or simplistic fix patterns for developers. CCS CONCEPTS • Software and its engineering → Software testing and debugging.

show abstract

An empirical study of the long duration of continuous integration builds

Cited by 53 publications

References 49 publications

What Helped, and what did not? An Evaluation of the Strategies to Improve Continuous Integration

What Helped, and what did not? An Evaluation of the Strategies to Improve Continuous Integration

An empirical characterization of bad practices in continuous integration

Understanding build issue resolution in practice: symptoms and fix patterns

Contact Info

Product

Resources

About