Automated migration of build scripts using dynamic analysis and search-based refactoring

Abstract

The efficiency of a build system is an important factor for developer productivity. As a result, developer teams have been increasingly adopting new build systems that allow higher build parallelization. However, migrating the existing legacy build scripts to new build systems is a tedious and error-prone process. Unfortunately, there is insufficient support for automated migration of build scripts, making the migration more problematic.We propose the first dynamic approach for automated migration of build scripts to new build systems. Our approach works in two phases. First, from a set of execution traces, we synthesize build scripts that accurately capture the intent of the original build. The synthesized build scripts are typically long and hard to maintain. Second, we apply refactorings that raise the abstraction level of the synthesized scripts (e.g., introduce functions for similar fragments). As different refactoring sequences may lead to different build scripts, we use a search-based approach that explores various sequences to identify the best (e.g., shortest) build script. We optimize search-based refactoring with partial-order reduction to faster explore refactoring sequences. We implemented the proposed two-phase migration approach in a tool called Metamorphosis that has been recently used at Microsoft.

Detecting Incorrect Build Rules

“…CloudMake [17] is a distributed build system by Microsoft which exposes an interface based on TypeScript to specify build definitions. As with all new build systems, this project faced adoption issues as well: the company already had millions of lines of code compiled using definitions based on another system.…”

Object-Oriented Build Automation -- A Case Study

“…One should note that Gradle [7] does boast being object-oriented but its model is very limited -as it proposes the Task and the Project concepts only. Build automation tools are very niche and overlooked [9] part of various research -occasionally appearing as a presentation of new frameworks, designed to solve specific tasks i.e incremental build with dynamic dependencies [10], dependencies optimization [11] or migration to other build systems [12]. Occasionally, some papers appear which reminds the industry either about the requirements for modern build systems [13,14,15] or about the dangers that old build systems possess [16] (in this case technical debt).…”

Identifying Bugs in Make and JVM-Oriented Builds

“…Such scripts may involve the compilation of source files, application testing, and the construction of software artifacts such as libraries and executables. The advent of Continuous Integration (ci) together with the complexity of modern software systems have made prominent two important properties related to automated builds: efficiency and reliability [Gligoric et al, 2014;Hilton et al, 2016;Vakilian et al, 2015;Visser et al, 2016]. To save computing resources and development time [Hilton et al, 2016;Licker and Rice, 2019], build tools must be capable of coping with complex systems quickly, but without sacrificing the reliability of the final deliverables.…”

“…To save computing resources and development time [Hilton et al, 2016;Licker and Rice, 2019], build tools must be capable of coping with complex systems quickly, but without sacrificing the reliability of the final deliverables. Following this direction, new build systems have emerged providing features such as parallelism [Bazel, 2020;Coetzee et al, 2011;Gligoric et al, 2014], caching [Gradle Inc., 2020a], incrementality [Erdweg et al, 2015;Konat et al, 2018], and the lazy retrieval of project dependencies [Celik et al, 2016].…”