Assisting Malware Analysis with Symbolic Execution: A Case Study

Baldoni, Roberto; Coppa, Emilio; D’Elia, Daniele Cono; Demetrescu, Camil

doi:10.1007/978-3-319-60080-2_12

Cited by 21 publications

(23 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To do this, we train M = 25 separate HMMs, where each HMM is trained on all samples for a particular API function. 13 For an arbitrary mth HMM model, we vectorize using the 12 Figure 3 also suggests that when the predictive model is wrong, the model tends to confuse the true API call with only a very small number of other calls. Indeed, a follow up analysis revealed that the generic API call deobfuscator included the correct API call in its top prediction 75.50% of the time, top 2 predictions 89.26% of the time, and top 3 predictions 92.38% of the time.…”

Section: A Methodologymentioning

confidence: 99%

“…Indeed, a follow up analysis revealed that the generic API call deobfuscator included the correct API call in its top prediction 75.50% of the time, top 2 predictions 89.26% of the time, and top 3 predictions 92.38% of the time. 13 Note that we are now treating each API call sample as an i.i.d sample from the population of some particular API function, rather than from a population of all API function, as in Experiment 1. number of arguments m appropriate for the corresponding API name. 14 In this way, we obtain M * (K + 3) = 325 features for each sample.…”

Section: A Methodologymentioning

confidence: 99%

“…e.g. malware analysis assistance [13], identifying trigger behavior in malware [14], augmenting fuzzing techniques [15], checking stability of patches and identifying bugs [16], etc.…”

Section: Background a Symbolic Executionmentioning

confidence: 99%

See 2 more Smart Citations

Towards Generic Deobfuscation of Windows API Calls

Kotov¹,

Wojnowicz²

2018

Proceedings 2018 Workshop on Binary Analysis Research

View full text Add to dashboard Cite

A common way to get insight into a malicious program's functionality is to look at which API functions it calls. To complicate the reverse engineering of their programs, malware authors deploy API obfuscation techniques, hiding them from analysts' eyes and anti-malware scanners. This problem can be partially addressed by using dynamic analysis; that is, by executing a malware sample in a controlled environment and logging the API calls. However, malware that is aware of virtual machines and sandboxes might terminate without showing any signs of malicious behavior. In this paper, we introduce a static analysis technique allowing generic deobfuscation of Windows API calls. The technique utilizes symbolic execution and hidden Markov models to predict API names from the arguments passed to the API functions. Our best prediction model can correctly identify API names with 87.60% accuracy.Arguments 5, 6, 7 and 9 are pre-defined constants (permission flags, attributes etc.) and can only take a finite and small number of potential values (it's also partially true for

show abstract

Section: A Methodologymentioning

confidence: 99%

Section: A Methodologymentioning

confidence: 99%

See 1 more Smart Citation

Towards Generic Deobfuscation of Windows API Calls

Kotov¹,

Wojnowicz²

2018

Proceedings 2018 Workshop on Binary Analysis Research

View full text Add to dashboard Cite

show abstract

“…The ability of ANGR to reason on binary code lifted to platform-agnostic VEX IR makes it an extremely powerful, versatile tool for security researchers that exploit symbolic execution with the ultimate goal of, for example, finding vulnerabilities [5], bypassing authentication checks in 20 of 35 L. Borzacchiello ET AL. device firmware [26] or dissecting malware [27]. Although ANGR integrates several state-of-the-art program analysis techniques such as veritesting [4] to improve scalability, it still struggles when analysing real-world programs.…”

Section: Application Domainsmentioning

confidence: 99%

Memory models in symbolic execution: key ideas and new thoughts

Borzacchiello

Coppa

D’Elia

et al. 2019

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

Symbolic execution is a popular program analysis technique that allows seeking for bugs by reasoning over multiple alternative execution states at once. As the number of states to explore may grow exponentially, a symbolic executor may quickly run out of space. For instance, a memory access to a symbolic address may potentially reference the entire address space, leading to a combinatorial explosion of the possible resulting execution states. To cope with this issue, state-of-the-art executors either concretize symbolic addresses that span memory intervals larger than some threshold or rely on advanced capabilities of modern satisfiability modulo theories solvers. Unfortunately, concretization may result in missing interesting execution states, for example, where a bug arises, while offloading the entire problem to constraint solvers can lead to very large query times. In this article, we first contribute to systematizing knowledge about memory models for symbolic execution, discussing how four mainstream symbolic executors deal with symbolic addresses. We then introduce MEMSIGHT, a new approach to symbolic memory that reduces the need for concretization: rather than mapping address instances to data as previous approaches do, our technique maps symbolic address expressions to data, maintaining the possible alternative states resulting from the memory referenced by a symbolic address in a compact, implicit form. Experiments on prominent programs show that MEMSIGHT, which we implemented in both ANGR and KLEE, enables the exploration of states that are unreachable for memory models that perform concretization and provides a performance level comparable with memory models relying on advanced solver theories. MEMORY MODELS IN SYMBOLIC EXECUTION: KEY IDEAS AND NEW THOUGHTS3 of 35 used in the context of state-of-the-art symbolic executors. A crucial design goal for our approach is that it should be general enough not only to support bug detection but also to be valuable in application contexts where users are interested in possible consequences of these bugs, for example, understanding how they can be exploited.Our approach, which we call MEMSIGHT, natively accounts for merging [3,4], a mainstream performance enabler in symbolic executors. We integrated MEMSIGHT into the state-of-the-art symbolic executors ANGR [5] and KLEE [6]: an experimental evaluation shows that MEMSIGHT allows for broader state exploration on prominent benchmarks analysed with ANGR, while it provides valuable run-time speedups when symbolically executing real-world programs under KLEE.A preliminary version [7] of this work appeared in the Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2017), where we introduced the main idea behind MEMSIGHT (Section 3) and presented a preliminary experimental evaluation related to MEMSIGHT when integrated into ANGR (Section 5.2.2). ‡ For our example to be meaningful, we assume that b resides in memory rather than in a CPU register. § In SMT-LIB [9], the theories of...

show abstract

“…Previous research has explored static analysis techniques to ease RAT dissection: in particular, [2] proposes symbolic execution to reveal and analyze the commands supported by a RAT without requiring the presence of the server counterpart. The output is a collection of execution traces enriched with symbolic constraints on the data buffers exchanged throughout communications.…”

Section: Introductionmentioning

confidence: 99%

Reconstructing C2 Servers for Remote Access Trojans with Symbolic Execution

Borzacchiello

Coppa

D’Elia

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

The analysis of a malicious piece of software that involves a remote counterpart that instructs it can be troublesome for security professionals, as they may have to unravel the communication protocol in use to figure out what actions can be carried out on the victim's machine. The possibility to recur to dynamic analysis hinges on the availability of an active remote counterpart, a requirement that may be difficult to meet in several scenarios. In this paper we explore how symbolic execution techniques can be used to synthesize a command-and-control server for a remote access trojan, enabling in-vivo analysis by malware analysts. We evaluate our ideas against two real-world malware instances.

show abstract

Assisting Malware Analysis with Symbolic Execution: A Case Study

Cited by 21 publications

References 19 publications

Towards Generic Deobfuscation of Windows API Calls

Towards Generic Deobfuscation of Windows API Calls

Memory models in symbolic execution: key ideas and new thoughts

Reconstructing C2 Servers for Remote Access Trojans with Symbolic Execution

Contact Info

Product

Resources

About