Automated Decomposition of Build Targets

Abstract

Abstract-A (build) target specifies the information that is needed to automatically build a software artifact. This paper focuses on underutilized targets-an important dependency problem that we identified at Google. An underutilized target is one with files not needed by some of its dependents. Underutilized targets result in less modular code, overly large artifacts, slow builds, and unnecessary build and test triggers. To mitigate these problems, programmers decompose underutilized targets into smaller targets. However, manually decomposing a target is tedious and error-prone. Although we prove that finding the best target decomposition is NP-hard, we introduce a greedy algorithm that proposes a decomposition through iterative unification of the strongly connected components of the target. Our tool found that 19,994 of 40,000 Java library targets at Google can be decomposed to at least two targets. The results show that our tool is (1) efficient because it analyzes a target in two minutes on average and (2) effective because for each of 1,010 targets, it would save at least 50% of the total execution time of the tests triggered by the target.

Identifying and understanding header file hotspots in C/C++ build processes

“…Underutilized targets may trigger updates to those targets that depend upon them, even if those updates are not strictly necessary. By dividing underutilized targets into smaller, independent ones, Vakilian et al (2015) are able to improve build performance. We believe that our approach is complementary to the approach of Vakilian et al (2015) in two ways: (1) our technique includes the change frequency dimension, which could be used to reduce the scope of the target refactoring to those targets that really make a difference in day-to-day development, and (2) applying the notion of decomposing underutilized targets to hotspot files may be an interesting way to address build hotspots.…”

“…By dividing underutilized targets into smaller, independent ones, Vakilian et al (2015) are able to improve build performance. We believe that our approach is complementary to the approach of Vakilian et al (2015) in two ways: (1) our technique includes the change frequency dimension, which could be used to reduce the scope of the target refactoring to those targets that really make a difference in day-to-day development, and (2) applying the notion of decomposing underutilized targets to hotspot files may be an interesting way to address build hotspots. Furthermore, while the approach of Vakilian et al (2015) focuses on refactoring build specifications, our technique focuses on files, components, and subsystems that would benefit from build optimization effort.…”

“…For example, Vakilian et al (2015) devise a technique to detect and refactor underutilized targets, i.e., build targets that are only partially used by the targets that depend upon them. Underutilized targets may trigger updates to those targets that depend upon them, even if those updates are not strictly necessary.…”

Automated migration of build scripts using dynamic analysis and search-based refactoring

“…A few projects have explored maintenance of build scripts [16,37,64,66]. Formiga [37] supports simple renaming and removal of targets.…”

“…Vakilian et. al [66] developed tools for decomposing Google build specifications. Although valuable, previously proposed refactorings are not suitable for improving synthesized build scripts.…”