Practical fault detection in puppet programs

Abstract

Puppet is a popular computer system configuration management tool. By providing abstractions that model system resources it allows administrators to set up computer systems in a reliable, predictable, and documented fashion. Its use suffers from two potential pitfalls. First, if ordering constraints are not correctly specified whenever a Puppet resource depends on another, the nondeterministic application of resources can lead to race conditions and consequent failures. Second, if a service is not tied to its resources (through the notification construct), the system may operate in a stale state whenever a resource gets modified. Such faults can degrade a computing infrastructure's availability and functionality.We have developed an approach that identifies these issues through the analysis of a Puppet program and its system call trace. Specifically, a formal model for traces allows us to capture the interactions of Puppet resources with the file system. By analyzing these interactions we identify (1) resources that are related to each other (e.g., operate on the same file), and (2) resources that should act as notifiers so that changes are correctly propagated. We then check the relationships from the trace's analysis against the program's dependency graph: a representation containing all the ordering constraints and notifications declared in the program. If a mismatch is detected, our system reports a potential fault.We have evaluated our method on a large set of popular Puppet modules, and discovered 92 previously unknown issues in 33 modules. Performance benchmarking shows that our approach can analyze in seconds real-world configurations with a magnitude measured in thousands of lines and millions of system calls. CCS CONCEPTS• Software and its engineering → Software reliability; Software testing and debugging; File systems management.

GLITCH: Automated Polyglot Security Smell Detection in Infrastructure as Code

“…Schwarz et al [24] picked smells from the catalog proposed by Sharma et al [26] and convert them into detection rules for Foodcritic, a static code analysis tool designed for Chef. Sotiropoulos et al [27] propose a tool for detecting faults regarding ordering violations and notifiers in Puppet scripts. Lepillet et al [10] propose Häyä, a tool that uses dataflow graph analysis to detect intra-update sniping vulnerabilities in CloudFormation templates.…”

“…Finally, some analysis tools for IaC use intermediate representations [6,25,27] to describe file-system manipulations done by IaC scripts. Shambaugh et al [25] translated IaC scripts to the intermediate representation by mapping types of resources to their filesystem operations.…”

“…Shambaugh et al [25] translated IaC scripts to the intermediate representation by mapping types of resources to their filesystem operations. Sotiropoulos et al [27] used system calls executed by each resource in a IaC script to automatically map the resources to the filesystem operations, which were represented in the intermediate language. To the best of our knowledge, our work is the first that translates scripts of different IaC technologies into an intermediate representation.…”

“…This challenge is about extending GLITCH to support more IaC technologies and to detect more vulnerabilities. For example, it would be interesting to extend GLITCH to support Terraform and to support the detection of faults regarding ordering violations [27] or intra-update sniping vulnerabilities [10].…”

Andromeda: A Dataset of Ansible Galaxy Roles and Their Evolution

“…A number of IaC-related works have published their evaluation datasets [3], [4], [8]- [10]. However, these datasets only contain a relatively low number of projects, and consist of mostly analysis results.…”