What is There About DevOps Assessment? A Systematic Mapping

Abstract

DevOps has been established as a framework used by software development companies seeking to set mechanisms to automate their development processes. Consequently, over the last decade, many companies have adopted DevOps to support their project’s development process and perform continuous improvement tasks to ensure that it is applied correctly. To achieve this, companies are looking for solutions that allow them to evaluate the degree of implementation of DevOps in their internal processes. In this sense, the objective of this study focuses on identifying, through a systematic mapping of the literature, the mechanisms used to assess DevOps in software development companies. According to the above, the current state of knowledge related to the proposal of processes, models, techniques, tools, and methodological guides is presented to conduct the DevOps assessment. As a result, it is noted that there are multiple methodological solutions that seek to assess DevOps; however, a high degree of heterogeneity was evidenced in the identified solutions, resulting in the need to establish a clear framework that serves as the basis for proposing a generic, structured, and unambiguous DevOps assessment model applicable to software companies.

Metrics Model to Complement the Evaluation of DevOps in Software Companies

“…After executing a systematic mapping of the literature (SML), as reported in [35], it was analyzed to identify the solutions proposed by different authors in relation to the definition of processes, models, techniques and/or or tools to evaluate DevOps in software companies. Three types of studies were identified: (i) exploratory studies, (ii) methodological solutions, and (iii) tools.…”

“…However, adopting DevOps in software companies is not a simple task [33], to minimize the risk of error in its adoption, they must establish mechanisms that allow quantifying how it is applied in their projects and identify improvement opportunities to fine-tune their practices and improve their internal processes [34]. The efforts and proposals related to the evaluation of DevOps in software companies were identified with a systematic mapping of the literature carried out in [35]. Two mechanisms were used to define methodological solutions (models, metrics, certification standards) and tools developed by active players in the industry that seek to assess DevOps in multiple ways.…”

“…Although the analyzed solutions follow the same objective: "assess the degree of DevOps capacity, maturity and/or competence", they have different perceptions, scopes and, in some cases, they are ambiguous. Likewise, the solutions described in [35] establish "what" to do; however, they do not define "how" to implement the proposed practices, which can cause confusion when applying DevOps in software companies. Besides, there are studies related to the evaluation of DevOps in companies of different sizes, most of them focus on large and medium-sized companies and leave aside small and micro software companies.…”