Binary Lifter Evaluation

Abstract

Binary rewriting gives software developers, consumers, attackers, and defenders the ability to modify both their own and third-party software for instrumentation, customization, optimization, and hardening. Unfortunately, the practical limitations of binary rewriting tools are often glossed over in academic publications in this field, making it unclear to users if and when these tools will work in practice. This, among other challenges, has prohibited the widespread adoption of binary rewriting tools. To address this shortcoming, we collect eight popular binary rewriting tools and assess their generality across a broad range of input binary classes and the functional reliability of the resulting rewritten binaries. Additionally, we evaluate the performance of the rewriting tools themselves as well as the rewritten binaries they produce. We also identify features that are predictive of rewriting success and show that a simple decision tree model trained on these features can accurately predict whether a particular tool can rewrite a target binary.The goal of this broad evaluation is to provide a state of the practice for binary rewriting tools. We hope our findings will inform potential users of binary rewriting, support binary rewriting tool developers, and set a shared context for future research in this area. The binary rewriters, our corpus of 3344 sample binaries, and the evaluation infrastructure itself are all freely available as opensource software.