A Brief Study on Build Failures in Continuous Integration: Causation and Effect

Jain, Romit; Singh, Saket Kumar; Mishra, Bharavi

doi:10.1007/978-981-13-0224-4_2

Cited by 6 publications

(2 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We did a preliminary investigation of these builds and found that these jobs with the same build identifier normally share the same build result and build duration. Thus, as many existing papers have done [14], [44], [26], we considered these jobs as a single build. After this filtering process, we obtained 82,427 builds from 100 projects (13,464 failing builds).…”

Section: A Data Setmentioning

confidence: 99%

“…Furthermore, many of our studied techniques [2], [20], [27], [33] were originally evaluated on TravisTorrent projects. Additionally, we extensively curated TravisTorrent, removing: toy projects following standard practice [25], [42], unusable projects for test-granularity techniques, and cancelled builds as in past work [14], [26], [44]. Finally, we also followed the advice in Gallaba et al 's study [14] to consider the nuance in the TravisTorrent dataset.…”

Section: A Internal Validitymentioning

confidence: 99%

See 1 more Smart Citation

What Helped, and what did not? An Evaluation of the Strategies to Improve Continuous Integration

Jin

Servant

2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Continuous integration (CI) is a widely used practice in modern software engineering. Unfortunately, it is also an expensive practice -Google and Mozilla estimate their CI systems in millions of dollars. There are a number of techniques and tools designed to or having the potential to save the cost of CI or expand its benefit -reducing time to feedback. However, their benefits in some dimensions may also result in drawbacks in others. They may also be beneficial in other scenarios where they are not designed to help. In this paper, we perform the first exhaustive comparison of techniques to improve CI, evaluating 14 variants of 10 techniques using selection and prioritization strategies on build and test granularity. We evaluate their strengths and weaknesses with 10 different cost and time-tofeedback saving metrics on 100 real-world projects. We analyze the results of all techniques to understand the design decisions that helped different dimensions of benefit. We also synthesized those results to lay out a series of recommendations for the development of future research techniques to advance this area.

show abstract

Section: A Data Setmentioning

confidence: 99%