BAHK: Flexible Automated Binary Analysis Method with the Assistance of Hardware and System Kernel

Abstract

To protect core functions, applications often utilize the countermeasure techniques such as antidebugging to avoid analysis by outsiders, especially the malware. Dynamic binary instrumentation is commonly used in the analysis of binary programs. However, it can be easily detected and has stability and applicability problems as it involves program rewriting and just-in-time compilation. This paper proposes a new lightweight analysis method for binary programs with the assistance of hardware features and the operating system kernel, named BAHK, which can automatically analyze the target program by stealth and has wide applicability. With the support of underlying infrastructures, this paper designs several optimization strategies and specific analysis approaches at instruction level to reduce the impact of fine-grained analysis on the performance of target program so that it can be well applied in practice. The experimental results show that the proposed method has good stealthiness, low memory consumption, and positive user experience. In some cases, it shows better analysis performance than the traditional dynamic binary instrumentation method. Finally, the real case studies further show its feasibility and effectiveness.

Lightweight and Efficient Hypervisor-Based Dynamic Binary Instrumentation and Analysis Method

“…For the analysis under the proposed framework, only a new basic block is executed by the program, the virtualization exception will be generated and handled in most cases, and the total number of exceptions is in a lower range, so the exception impact on the runtime performance is small. In fact, frequent virtualization exceptions can also have a significant impact on the execution of the target program, which has been studied and discussed in the previous work [31]. In Table I, it can also be observed that although the target program creates many threads and loads a lot of modules during the execution, it does not result in a significant impact on the interception efficiency.…”

“…Similar to the traditional method, The proposed method first needs to intercept the execution of the target program, but the difference is that here we accomplish the interception of code execution by controlling the access permissions of corresponding code pages of the target program based on the EPT mechanism [26] [31]. In fact, the processor with EPT mechanism has been widely available in commercial products, and many studies have applied its features to program debugging analysis, software protection, and so on [26][30] [32].…”

“…However, frequent virtualization exceptions will also cause a sharp decline in the performance of the system and program [31], so we use the method called "redirection on exception" to avoid this situation. In the handling of an exception, the basic block of the program is used as a unit to parse and rewrite.…”