An Empirical Evaluation of an Unpacking Method Implemented with Dynamic Binary Instrumentation

Abstract

SUMMARYMany malicious programs we encounter these days are armed with their own custom encoding methods (i.e., they are packed) to deter static binary analysis. Thus, the initial step to deal with unknown (possibly malicious) binary samples obtained from malware collecting systems ordinarily involves the unpacking step. In this paper, we focus on empirical experimental evaluations on a generic unpacking method built on a dynamic binary instrumentation (DBI) framework to figure out the applicability of the DBI-based approach. First, we present yet another method of generic binary unpacking extending a conventional unpacking heuristic. Our architecture includes managing shadow states to measure code exposure according to a simple byte state model. Among available platforms, we built an unpacking implementation on PIN DBI framework. Second, we describe evaluation experiments, conducted on wild malware collections, to discuss workability as well as limitations of our tool. Without the prior knowledge of 6029 samples in the collections, we have identified at around 64% of those were analyzable with our DBI-based generic unpacking tool which is configured to operate in fully automatic batch processing. Purging corrupted and unworkable samples in native systems, it was 72%.

An Original Entry Point Detection Method with Candidate-Sorting for More Effective Generic Unpacking

“…Kawakoya et al [10] focus on memory access 'write', 'read', and 'execute' of packed programs to detect the OEP, and Jeong et al [24] focus on entropy scores in each section of a packed program on the memory to do so. Kim et al [11] focus on a write-execute transition to spot more likely OEP candidates. Their system stores every written instruction, and it searches for a sequence of written instructions that have been executed successively.…”