Large-Scale Automated Refactoring Using ClangMR

Abstract

Maintaining large codebases can be a challenging endeavour. As new libraries, APIs and standards are introduced, old code is migrated to use them. To provide as clean and succinct an interface as possible for developers, old APIs are ideally removed as new ones are introduced. In practice, this becomes difficult as automatically finding and transforming code in a semantically correct way can be challenging, particularly as the size of a codebase increases.In this paper, we present a real-world implementation of a system to refactor large C++ codebases efficiently. A combination of the Clang compiler framework and the MapReduce parallel processor, ClangMR enables code maintainers to easily and correctly transform large collections of code. We describe the motivation behind such a tool, its implementation and then present our experiences using it in a recent API update with Google's C++ codebase.

Lessons from building static analysis tools at Google

“…Concurrent with FindBugs experimentation, the C++ workflow at Google was improving with the addition of new checks to the Clang compiler. The Clang team implemented new compiler checks, along with suggested fixes, then used ClangMR 38 to run the updated compiler in a distributed way over the entire Google codebase, refine checks, and programmatically fix all existing instances of a problem in the codebase. Once the codebase was cleansed of an issue, the Clang team enabled the new diagnostic as a compiler error (not a warning, which the Clang team found Google developers ignored) to break the build, a report difficult to disregard.…”

SapFix: Automated End-to-End Repair at Scale

“…However, Tricorder fixes are typically manually specified along with the analysis check, whereas Sapfixes are detected and checked by automatically-constructed tests as with SAPFIX. Automated refactoring has also been widely studied [60] and has found deployment at scale in industry [61]. However, refactoring seeks to apply known-to-be semantically safe changes; essentially altering syntax without disrupting semantics.…”

Understanding Characteristics and Structural Effects of Batch Refactorings in Practice

“…Major companies like Google (71) and Microsoft (35) have employed code refactoring for leveraging the internal quality of their software projects. As a software development practice, code refactoring consists of applying two or more transformations on the code structure of a given project (25).…”

“…Therefore, characterizing the actual effect of batches on smell introduction and removal is essential to guide developers in enhancing code structures via code refactoring. In fact, code refactoring has been largely employed by the industry for removing poor code structures (35,46,60,71). In this context, a previous study (7) has presented the very first study aimed to empirically assess the batch effect on code smell introduction and removal.…”

“…Code refactoring consists of applying one or more transformations on the code structure of a software project (25). Refactoring has been largely employed by developers, in major companies such as Google (71) and Microsoft (35), to remove poor code structures that represent threats to code maintenance (25). Software projects are usually considered maintainable when their code structures are easy to understand and modify (72).…”