Instruction embedding for improved obfuscation

LeDoux, Charles; Sharkey, Michael; Primeaux, Brandon; Miles, Craig

doi:10.1145/2184512.2184543

Cited by 16 publications

(12 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Obfuscation [5] is a traditional method of software protection, such as control flow expansion [6], garbage code insertion [7], instruction deformation [8], binary code encryption and packaging [9], and virtualization obfuscation [10]. These code obfuscation techniques are now common in malware, which make it difficult to discover the true logic of the program and give the security analyst an incredibly hard time.…”

Section: Code Obfuscationmentioning

confidence: 99%

VMGuards: A Novel Virtual Machine Based Code Protection System with VM Security as the First Class Design Concern

Tang

et al. 2018

Applied Sciences

View full text Add to dashboard Cite

Process-level virtual machine (PVM) based code obfuscation is a viable means for protecting software against runtime code tampering and unauthorized code reverse engineering. PVM-based approaches rely on a VM to determine how instructions of the protected code region are scheduled and executed. Therefore, it is crucial to protect the VM against runtime code tampering that alters the instructions and behavior of the VM. This paper presents VMGuards, a novel PVM-based code protection system that puts the security of VM as the first class design concern. Our approach advances prior work by promoting security of the VM as the first class design constraint. We achieve this by introducing two new instruction sets to protect the internal implementations of critical code segments and the host runtime environment where the VM runs in. Our new instruction sets not only have an identical code structure as standard virtual instructions, but also provide additional information to allow the VM to check whether the critical internal implementation or the runtime environment is affected. We evaluate our approach by using a set of real-life applications. Experimental results show that our approach provides stronger and more fine-grained protection when compared to the state-of-the-art with little extra overhead.

show abstract

Section: Code Obfuscationmentioning

confidence: 99%

VMGuards: A Novel Virtual Machine Based Code Protection System with VM Security as the First Class Design Concern

Tang

et al. 2018

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Linn et al designed an obfuscation to thwart linear sweep and recursive traversal static disassembly algorithms [23], which introduced junk instructions, jump tables, and opaque predicates to disrupt the static disassembly process. Ledoux et al proposed an instruction embedding method to improve instruction overlapping obfuscation [24]. Batchelder et al presented layout obfuscation algorithm to make Java decompilers fail to produce legal source code or crash completely [25].…”

Section: Related Workmentioning

confidence: 99%

An Iteration Obfuscation Based on Instruction Fragment Diversification and Control Flow Randomization

Liu¹,

Xie²,

Lu³

et al. 2016

IJCTE

View full text Add to dashboard Cite

Abstract-As the control flow graph can reflect the logic structure of programs, static and dynamic reverse methods are used to analyze the logic structure and instruction sequence, and the existing methods of control flow obfuscation have low potency to resist reverse attacks. To solve this problem, we propose an obfuscation method based on instruction fragment diversification and control flow randomization, diversified instruction fragments are generated by various equivalent transformation rules, and random functions are used to select one execution path from the multi-way branches of programs, then programs are iteratively obfuscated. Experiments and analysis show that diversified instruction fragments and multi-way branches can increase the difficulty of static reverse analysis, random selection for multi-way branches will increase the difficulty of dynamic instruction tracing, and iterative transformation for many times enhances the complexity of control flow graph.Index Terms-Code obfuscation, iterative transformation, instruction fragment diversification, control flow randomization.

show abstract

“…Each technique uses a different method to confuse the automated disassembly tool in successfully disassembling the binary program. Some of the binary obfuscation methods like signal based obfuscation [15], dynamic code mutation [16], dummy variables based obfuscation [17], mimimorphic obfuscation [18], virtual machine based obfuscation [19] and instruction embedding obfuscation [20] are discussed in this section.…”

Section: Related Workmentioning

confidence: 99%

“…Embedding instructions in the program execution path [20] is another mechanism used to achieve obfuscation. The basic idea of this method is embedding hexadecimal bytes in the program execution path so that these bytes overlap with the assembly instructions in the original code creating new instructions.…”

Section: Related Workmentioning

confidence: 99%

Obfuscation by code fragmentation to evade reverse engineering

Balachandran

Emmanuel

2014

2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

View full text Add to dashboard Cite

Software distributed can be reverse engineered using software analyzing tools, thereby letting an adversary understand the logic of the program and finding the vulnerabilities in the program. Control flow obfuscation is one of the techniques that can obscure the program so that the reverse engineering tools get an erroneous result while analyzing the program. In this paper, we propose a binary obfuscation algorithm that makes it difficult to reverse engineer the programs. The basic idea is to fragment the binary code by moving instruction sets from various parts of the program to a new code block. The control flow, to and from the new code block, is camouflaged using dynamic instruction modification during runtime. Experimental results indicate that the algorithm performs well against reverse engineering attacks by standard tools.

show abstract

Instruction embedding for improved obfuscation

Cited by 16 publications

References 4 publications

VMGuards: A Novel Virtual Machine Based Code Protection System with VM Security as the First Class Design Concern

VMGuards: A Novel Virtual Machine Based Code Protection System with VM Security as the First Class Design Concern

An Iteration Obfuscation Based on Instruction Fragment Diversification and Control Flow Randomization

Obfuscation by code fragmentation to evade reverse engineering

Contact Info

Product

Resources

About