On the Effectiveness of Tools to Support Infrastructure as Code: Model-Driven Versus Code-Centric

Abstract

Infrastructure as Code (IaC) is an approach for infrastructure automation that is based on software development practices. The IaC approach supports code-centric tools that use scripts to specify the creation, updating and execution of cloud infrastructure resources. Since each cloud provider offers a different type of infrastructure, the definition of an infrastructure resource (e.g., a virtual machine) implies writing several lines of code that greatly depend on the target cloud provider. Model-driven tools, meanwhile, abstract the complexity of using IaC scripts through the high-level modeling of the cloud infrastructure. In a previous work, we presented an infrastructure modeling approach and tool (Argon) for cloud provisioning that leverages model-driven engineering and supports the IaC approach. The objective of the present work is to compare a model-driven tool (Argon) with a well-known code-centric tool (Ansible) in order to provide empirical evidence of their effectiveness when defining the cloud infrastructure, and the participants' perceptions when using these tools. We, therefore, conducted a family of three experiments involving 67 Computer Science students in order to compare Argon with Ansible as regards their effectiveness, efficiency, perceived ease of use, perceived usefulness, and intention to use. We used the AB/BA crossover design to configure the individual experiments and the linear mixed model to statistically analyze the data collected and subsequently obtain empirical findings. The results of the individual experiments and metaanalysis indicate that Argon is more effective as regards supporting the IaC approach in terms of defining the cloud infrastructure. The participants also perceived that Argon is easier to use and more useful for specifying the infrastructure resources. Our findings suggest that Argon accelerates the provisioning process by modeling the cloud infrastructure and automating the generation of scripts for different DevOps tools when compared to Ansible, which is a code-centric tool that is greatly used in practice. INDEX TERMS Infrastructure as code, DevOps, model-driven engineering, controlled experiments, crossover design, linear mixed model. JULIO SANDOBALÍN received the M.Sc. degree in software engineering from the Universitat Politècnica de València, Spain, in 2014, where he is currently pursuing the Ph.D. degree with the Software Engineering and Information Systems Research Group. His research activities and interests include a wide range of subjects related to model-driven engineering, cloud computing, DevOps, and empirical software engineering. His research is currently focused on the continuous delivery of cloud resources based on model-driven engineering and DevOps.

An empirical study on visual programming docker compose configurations

“…However, they mostly focus on resolving the vendor lock-in issue and facilitating work in multi-cloud settings. However, Sandobalin et al [14,15] already propose the development of IaC with high abstraction level programming, being this close to our work focus.…”

“…Sandobalin et al [14,15] proposed ARGON as a DevOps support tool leveraging IaC practices and its benefits for infrastructure provisioning while simultaneously minimizing its drawbacks through a model-driven approach. The project defines a DSL to model infrastructure independently of the provider, later used to generate the target tool script.…”

Critical Challenges to Adopt DevOps Culture in Software Organizations: A Systematic Review

“…Annual 'State of DevOps' reports reveals that as DevOps practices spread, the number of DevOps teams has grown from 19% in 2015 to 22% in 2016 to 27% in 2017 [6,19]. Organizations have massively embraced DevOps principles due to current customers' demand for highly available, continuous-release, and high-value applications that are useable anytime, anywhere at any platform [20]. Leading ventures such as Google, Netflix, Amazon, LinkedIn, Spotify, Flicker, and Etsy have adopted DevOps practices to release software with a higher pace and better quality [21][22][23][24].…”

Scientific Workflow Execution Using a Dynamic Runtime Model

“…CAMEL also introduces a models at runtime approach utilizing the Cloudiator toolkit [116] to deploy and manage the lifecycle of modelled cloud deployments. The tool An infRastructure modelinG tool for clOud provisioniNg (ARGON) [117,118] also provides a DSL for cloud environments and uses model-driven techniques to generate IaC artifacts in order to orchestrate modeled infrastructures. The Saloon framework [119,120] is used to form a software product line.…”

“…Abstraction layers and metamodels are often used to teach about domains as they provide an abstract description language that is typically supported by an accompanied visual notation. While the UML standard is the de-facto staple for teaching about software architectures and design, cloud languages also have been proven useful to simplify the utilization of cloud deployments [117]. In this section, we discuss the potential impact of our approach for educational purposes.…”