Selection of DevOps best test practices: A hybrid approach using ISM and fuzzy TOPSIS analysis

Abstract

Testing is a complex phase in DevOps process due to need of an automated process that provides feedback at different strategies of continuous development and operations pipeline. Software organization face several challenges during the testing phase due to lack of understanding on testing best practices for the DevOps paradigm. The objective of this study is to prioritize DevOps best testing practices, which can facilitate the selection of testing practices during DevOps process. To perform this research, we have extended the work done by Hornbeek, using the 15 DevOps testing practices discussed in his study. First, we categorize the test practices against culture, automation, lean, measurement, and sharing (CALMS) pillars of DevOps adoption principles. Next, a questionnaire‐based survey was conducted to collect feedback from industry practitioners on the DevOps test practices and their categorization against CALMS criteria. Finally, we applied Interpretive Structure Modeling (ISM) to find the interrelationship between CALMS criteria, and fuzzy TOPSIS was used to prioritize the DevOps test practices that will assist practitioners to better manage the testing activities during DevOps process.

Evaluation of Requirement Engineering Best Practices for Secure Software Development in GSD: An ISM Analysis

“…Interaction between RE practices and the key knowledge areas is determined using the ISM method. Many academic researchers have used a similar methodology to investigate the contextual interaction of elements [40][41][42][43]47]. ISM method helps in presenting a complex system in a simplified way, provides an interpretation of the embedded object, and transforms unclear and poorly articulated mental models of systems into visible, well-defined models, thereby helping in answering what and how in theory, building facilitates the identification of the structure within a system [40][41][42][43]47].…”

“…Many academic researchers have used a similar methodology to investigate the contextual interaction of elements [40][41][42][43]47]. ISM method helps in presenting a complex system in a simplified way, provides an interpretation of the embedded object, and transforms unclear and poorly articulated mental models of systems into visible, well-defined models, thereby helping in answering what and how in theory, building facilitates the identification of the structure within a system [40][41][42][43]47]. A structural-self-interaction matrix (SSIM), as provided in the following sections, is required to develop the contextual interaction between the criteria.…”

“…The model uses clear patterns to show the structure's complexity through graphs and words [38]. The ISM approach assists in finding various relationships in complicated situations when the relationship consists of different variables [39,40]. Several scholars have used this method to construct a better conceptual framework for the system under examination [41][42][43][44][45].…”

“…Azeem et al [47] applied the ISM approach to study the relationships among the core categories of the challenges in DevSecOps. Similarly, other researchers also use the ISM approach to find the interrelationship between the DevOps testing process [41] and best test practices [40].…”

Evaluation of requirement engineering best practices for secure software development in GSD: An ISM analysis

“…37 The ISM approach assists in finding various relationships in complicated situations when the relationship consists of different variables. 38 Several scholars have used this method to construct a better conceptual framework for the system under examination. [39][40][41][42][43] Figure 2 explains the steps taken by the ISM methodology to determine the connection between RE security practices categorization.…”