Grexmk

Abstract

Slow builds can be extremely costly; for example, one of our customers loses forty percent of their developers' productivity to waiting on builds. Build avoidance-building only what must be built to integrate a change-solves the problem but makes the build unreliable unless the build script captures all dependences of build outputs on sources. Therefore, in practice, integrating even a minor change often requires rebuilding a large software system from scratch.Our solution can be summarized as "trust but verify". Exploiting the fact that large software systems tend to be composed of loosely coupled parts, we semi-automatically divide a large, all-or-nothing build into many small "mini-builds", without changing the original build. Each mini-build declares its dependences, which are trusted while scheduling mini-builds to integrate a change but verified by executing each mini-build in a sandbox in which only its declared dependences are available.We implemented these ideas in a tool suite called Grexmk and applied Grexmk to two large all-or-nothing builds. On the first build, we demonstrated linear speedups to six build machines. On the second build, we sped up the average time for an incremental build by a factor of 1.2.

Practical fault detection in puppet programs

“…Then, we automatically ran every module separately through the include <module-name> statement. 3 We monitored the Puppet process and collected the system call trace of every program via strace. Through this process, we successfully applied 354 Puppet modules in total.…”

“…Analysis of system call traces has been widely used in the past, especially in the context of dependency inference P r e p r i n t for builds scripts. Many existing approaches are used for refactoring and testing [11], boosting performance [3,6] of builds, or detecting license inconsistencies in open-source projects [37]. As discussed in Section 3.1, most of them do not deal with the intricacies of system calls.…”