An empirical characterization of bad practices in continuous integration

Abstract

Continuous Integration (CI) has been claimed to introduce several benefits in software development, including high software quality and reliability. However, recent work pointed out challenges, barriers and bad practices characterizing its adoption. This paper empirically investigates what are the bad practices experienced by developers applying CI. The investigation has been conducted by leveraging semi-structured interviews of 13 experts and mining more than 2,300 Stack Overflow posts. As a result, we compiled a catalog of 79 CI bad smells belonging to 7 categories related to different dimensions of a CI pipeline management and process. We have also investigated the perceived importance of the identified bad smells through a survey involving 26 professional developers, and discussed how the results of our study relate to existing knowledge about CI bad practices. Whilst some results, such as the poor usage of branches, confirm existing literature, the study also highlights uncovered bad practices, e.g., related to static analysis tools or the abuse of shell scripts, and contradict knowledge from existing literature, e.g., about avoiding nightly builds. We discuss the implications of our catalog of CI bad smells for (i) practitioners, e.g., favor specific, portable tools over hacking, and do not ignore nor hide build failures, (ii) educators, e.g., teach CI culture, not just technology, and teach CI by providing examples of what not to do, and (iii) researchers, e.g., developing support for failure analysis, as well as automated CI bad smell detectors.

Towards a taxonomy of code review smells

“…Rompaey et al [68] defined the symptoms of poorly designed tests as test smells. Zampetti et al [69] categorized the bad practices followed in the continuous integration process. In the future, other steps/processes within the software development life cycle can be investigated to detect and avoid the smells.…”

Containing Malicious Package Updates in npm with a Lightweight Permission System

“…Developers typically hope that the community at large will find and report vulnerabilities quickly, but past attacks remained undetected for months or caused significant damage within short periods. Current static analysis and anomaly detection tools detect usually only very specific issues and produce many false alarms [18,31,66,80]. • Tracking known vulnerabilities: Many third-party services scan the dependency tree of Node.js applications for known vulnerabilities (e.g., Snyk.io, npm, GitHub).…”

Investigating the Interplay between Developers and Automation

“…I also discuss negative phenomena per CI practice and augment them with Duvall's catalog of CI practice anti-patterns [59]. Similarly to Zampetti et al [60], I use Duvall's anti-patterns to categorize incorrect applications of a practice in the sense that negative outcomes resulting from these anti-patterns could be considered risks or backlash. For the remainder of this section, benets refer to positive eects attributed to particular practices, while challenges refer to impediments to practice application as well as negative eects resulting from practice application.…”

“…Thus, it comes as no surprise that this codebase structure results in lengthy build times [61,63]. And while Zampetti et al [60] do discuss issues related to codebase structure, these are typically mentioned in a design context as opposed to how it may aect build time or complexity.…”