Continuous Delivery Practices in a Large Financial Organization

Abstract

Continuous Delivery is an agile software development practice in which developers frequently integrate changes into the main development line and produce releases of their software. An automated Continuous Integration infrastructure builds and tests these changes. Claimed advantages of CD include early discovery of (integration) errors, reduced cycle time, and better adoption of coding standards and guidelines. This paper reports on a study in which we surveyed 152 developers of a large financial organization (ING Nederland), and investigated how they adopt a Continuous Integration and delivery pipeline during their development activities. In our study, we focus on topics related to managing technical debt, as well as test automation practices. The survey results shed light on the adoption of some agile methods in practice, and sometimes confirm, while in other cases, confute common wisdom and results obtained in other studies. For example, we found that refactoring tends to be performed together with other development activities, technical debt is almost always "self-admitted", developers timely document source code, and assure the quality of their product through extensive automated testing, with a third of respondents dedicating more than 50% of their time to do testing activities.

Test activities in the continuous integration and delivery pipeline

“…The remaining 43 publications often describe development in a smaller context, but also development of large‐scale software systems with different ways of integrating the product. Vassallo et al report experiences from a large financial organization, which develops and tests software in feature branches, and then test the complete product in the main track's continuous delivery pipeline. Vost presents the way of working in the automotive industry, ie, integration of precompiled binaries in ECUs into a complete vehicle.…”

“…Testing before deployment to production is also described in different ways. Some organizations seem to have a way of working, which is close to continuous deployment . Other papers describe how the software must be tested and stabilized on a release branch before a release …”

“…The characteristics of the Test Activity Stakeholders (TAS) model are in different ways corresponding to similar descriptions that can be found in literature. The three pipeline phases (feature or subsystem branches, a main track, and a release branch) are described in various ways in several research papers . The three phases in the TAS model (Early in the pipeline, Later in the pipeline, and Release pipeline) also correspond to how Humble and Farley describe a pipeline (in their often‐cited book).…”

“…A few publications (eg, Liu, Marijan, Rathod and Surve) describe one type of testing as better than others, but from one specific viewpoint. Four papers (Laghari et al, Mihindukulasooriya et al, Rathod and Surve, Vassallo et al) do in different ways discuss test approaches where different types of test activities are combined but do not discuss how different types of test activities can fulfill different purposes.…”

“…In a large‐scale implementation of continuous integration and continuous delivery, test activities are assembled into a pipeline, which splits the test process into multiple stages. This pipeline has been referred to with different terminology, eg, as “integration pipeline,” “deployment pipeline,” “continuous integration pipeline,” “continuous delivery pipeline,” “continuous integration and delivery pipeline.”…”