Automated migration of build scripts using dynamic analysis and search-based refactoring

Gligoric, Milos; Schulte, Wolfram; Prasad, Chandra; Velzen, Danny van; Narasamdya, Iman; Livshits, Benjamin

doi:10.1145/2714064.2660239

Cited by 19 publications

(11 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast to test execution, build script task execution typically produces side effects in the form of new files, e.g., files with object code in ELF format. Modern build systems such as Bazel [5] and CloudMake [21,27] can use other ways than timestamps to find modified files, e.g., comparing cryptographic hashes of files across revisions. Recent alternative build systems that aim to replace make include Tup [54] and Shake [43]; the former uses an explicit persisted dependency graph.…”

Section: Build Systemsmentioning

confidence: 99%

Practical Machine-Checked Formalization of Change Impact Analysis

Palmskog

Çelik

Gligoric

2020

Tools and Algorithms for the Construction and Analysis of Systems

Self Cite

View full text Add to dashboard Cite

Change impact analysis techniques determine the components affected by a change to a software system, and are used as part of many program analysis techniques and tools, e.g., in regression test selection, build systems, and compilers. The correctness of such analyses usually depends both on domain-specific properties and change impact analysis, and is rarely established formally, which is detrimental to trustworthiness. We present a formalization of change impact analysis with machine-checked proofs of correctness in the Coq proof assistant. Our formal model factors out domain-specific concerns and captures system components and their interrelations in terms of dependency graphs. Using compositionality, we also capture hierarchical impact analysis formally for the first time, which, e.g., can capture when impacted files are used to locate impacted tests inside those files. We refined our verified impact analysis for performance, extracted it to efficient executable OCaml code, and integrated it with a regression test selection tool, one regression proof selection tool, and one build system, replacing their existing impact analyses. We then evaluated the resulting toolchains on several open source projects, and our results show that the toolchains run with only small differences compared to the original running time. We believe our formalization can provide a basis for formally proving domain-specific techniques using change impact analysis correct, and our verified code can be integrated with additional tools to increase their reliability.

show abstract

Section: Build Systemsmentioning

confidence: 99%

Practical Machine-Checked Formalization of Change Impact Analysis

Palmskog

Çelik

Gligoric

2020

Tools and Algorithms for the Construction and Analysis of Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…Most analysis approaches are dynamic and actually execute the build to extract information. For example, van der Burg et al [43] dynamically detect which files are included in a build to check license compatibility, Metamorphosis [26] dynamically analyzes build system to migrate them, Dietrich [24] analyzes Kbuild based systems dynamically to derive presence conditions for source files, and our prior work, MkFault [21], combines runtime information with some structural analysis to localize build faults. However, dynamic approaches can only analyze one configuration at a time.…”

Section: Related Workmentioning

confidence: 99%

Four languages and lots of macros: analyzing autotools build systems

2017

View full text Add to dashboard Cite

Build systems are crucial for software system development, however there is a lack of tool support to help with their high maintenance overhead. GNU Autotools are widely used in the open source community, but users face various challenges from its hard to comprehend nature and staging of multiple code generation steps, often leading to low quality and error-prone build code. In this paper, we present a platform, AutoHaven, to provide a foundation for developers to create analysis tools to help them understand, maintain, and migrate their GNU Autotools build systems. Internally it uses approximate parsing and symbolic analysis of the build logic. We illustrate the use of the platform with two tools: ACSense helps developers to better understand their build systems and ACSniff detects build smells to improve build code quality. Our evaluation shows that AutoHaven can support most GNU Autotools build systems and can detect build smells in the wild.

show abstract

“…ElectricTest's technique for dependency detection is more related to work in Makefile (build) parallelization, such as EMake [33] or Metamorphisis [19]. These systems observe filesystem reads and writes for each step of the build process to detect dependencies between steps and infer which steps can be paralleled.…”

Section: Related Workmentioning

confidence: 99%

Efficient dependency detection for safe Java test acceleration

Bell

Kaiser

Melski³

et al. 2015

Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering

View full text Add to dashboard Cite

Slow builds remain a plague for software developers. The frequency with which code can be built (compiled, tested and packaged) directly impacts the productivity of developers: longer build times mean a longer wait before determining if a change to the application being built was successful. We have discovered that in the case of some languages, such as Java, the majority of build time is spent running tests, where dependencies between individual tests are complicated to discover, making many existing test acceleration techniques unsound to deploy in practice. Without knowledge of which tests are dependent on others, we cannot safely parallelize the execution of the tests, nor can we perform incremental testing (i.e., execute only a subset of an application's tests for each build). The previous techniques for detecting these dependencies did not scale to large test suites: given a test suite that normally ran in two hours, the best-case running scenario for the previous tool would have taken over 422 CPU days to find dependencies between all test methods (and would not soundly find all dependencies)-on the same project the exhaustive technique (to find all dependencies) would have taken over 10 300 years. We present a novel approach to detecting all dependencies between test cases in large projects that can enable safe exploitation of parallelism and test selection with a modest analysis cost.

show abstract

Automated migration of build scripts using dynamic analysis and search-based refactoring

Cited by 19 publications

References 58 publications

Practical Machine-Checked Formalization of Change Impact Analysis

Practical Machine-Checked Formalization of Change Impact Analysis

Four languages and lots of macros: analyzing autotools build systems

Efficient dependency detection for safe Java test acceleration

Contact Info

Product

Resources

About