Use and Misuse of Continuous Integration Features: An Empirical Study of Projects That (Mis)Use Travis CI

Gallaba, Keheliya; McIntosh, Shane

doi:10.1109/tse.2018.2838131

Cited by 54 publications

(35 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, build-time heavily depends on the different configurations of Travis CI (e.g., using commands in the wrong phase, etc. ), which may affect our results [15]. However, since our findings are based on a large number of projects, we expect the effect of the Travis CI configurations to be minimal.…”

Section: Construct Validitymentioning

confidence: 86%

See 1 more Smart Citation

Which Commits Can Be CI Skipped?

Mujahid

Shihab

Rilling

2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Continuous Integration (CI) frameworks such as Travis CI, automatically build and run tests whenever a new commit is submitted/pushed. Although there are many advantages in using CI, e.g., speeding up the release cycle and automating the test execution process, it has been noted that the CI process can take a very long time to complete. One of the possible reasons for such delays is the fact that some commits (e.g., changes to readme files) unnecessarily kick off the CI process. Therefore, the goal of this paper is to automate the process of determining which commits can be CI skipped. We start by examining the commits of 58 Java projects and identify commits that were explicitly CI skipped by developers. Based on the manual investigation of 1,813 explicitly CI skipped commits, we first devise an initial model of a CI skipped commit and use this model to propose a rule-based technique that automatically identifies commits that should be CI skipped. To evaluate the rule-based technique, we perform a study on unseen datasets extracted from ten projects and show that the devised rule-based technique is able to detect and label CI skip commits, achieving Areas Under the Curve (AUC) values between 0.56 and 0.98 (average of 0.73). Additionally, we show that, on average, our technique can reduce the number of commits that need to trigger the CI process by 18.16%. We also qualitatively triangulated our analysis on the importance of skipping the CI process through a survey with 40 developers. The survey results showed that 75% of the surveyed developers consider it to be nice, important or very important to have a technique that automatically flags CI skip commits. To operationalize our technique, we develop a publicly available prototype tool, called CI-SKIPPER, that can be integrated with any git repository and automatically mark commits that can be CI skipped.

show abstract

Section: Construct Validitymentioning

confidence: 86%

“…However, our technique is applicable to other CI services (e,.g. Circle CI 11 , AppVeyor 12 , and CodeShip 13 ) that allow developers to skip commit using the CI skip feature or other CI services that provide a plugin to add this CI skip feature, such as Jenkins 14 , Hudson 15 , and Bamboo 16 .…”

Section: External Validitymentioning

confidence: 99%

Which Commits Can Be CI Skipped?

Mujahid

Shihab

Rilling

2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…We evaluated the usefulness and relevance of the reports created by CI-ODOR in a second survey (6). For the survey, we conducted a case study, in which we analyzed the build logs of 36 projects (7).…”

Section: Methodology Overviewmentioning

confidence: 99%

“…Sharma et al, leveraged best practices associated with code quality management to assess configuration code quality and proposed a catalog of 13 implementation and 11 design configuration smells [23]. Recent work by Gallaba et al, [6] also investigated configuration smells and based on rules provided by linters (e.g., TRAVISLINT) they measure smells and derive automated fixes for them. Our smells have a different focus, because we look at process-related smells rather than at configuration issues.…”

Section: Related Workmentioning

confidence: 99%

Automated Reporting of Anti-Patterns and Decay in Continuous Integration

Vassallo

Proksch

Gall

et al. 2019

2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Continuous Integration (CI) is a widely-used software engineering practice. The software is continuously built so that changes can be easily integrated and issues such as unmet quality goals or style inconsistencies get detected early. Unfortunately, it is not only hard to introduce CI into an existing project, but it is also challenging to live up to the CI principles when facing tough deadlines or business decisions. Previous work has identified common anti-patterns that reduce the promised benefits of CI. Typically, these anti-patterns slowly creep into a project over time before they are identified. We argue that automated detection can help with early identification and prevent such a process decay. In this work, we further analyze this assumption and survey 124 developers about CI anti-patterns. From the results, we build CI-ODOR, a reporting tool for CI processes that detects the existence of four relevant anti-patterns by analyzing regular build logs and repository information. In a study on the 18,474 build logs of 36 popular JAVA projects, we reveal the presence of 3,823 high-severity warnings spread across projects. We validate our reports in a survey among 13 original developers of these projects and through general feedback from 42 developers that confirm the relevance of our reports.

show abstract

“…Previous works have proposed approaches for detecting misuses of specific configuration options. Gallaba et al, [9] achieved a high user acceptance when pointing out the misuse of four different configuration options, like executing commands in the wrong build step. Rahman et al [36] focused on security-related issues and Sharma et al [38] on Infrastructure-as-Code (IaC) smells.…”

Section: Introductionmentioning

confidence: 99%