Malware detection using assembly and API call sequences

Abstract

One of the major problems concerning information assurance is malicious code. To evade detection, malware has also been encrypted or obfuscated to produce variants that continue to plague properly defended and patched networks with zero day exploits. With malware and malware authors using obfuscation techniques to generate automated polymorphic and metamorphic versions, antivirus software must always keep up with their samples and create a signature that can recognize the new variants. Creating a signature for each variant in a timely fashion is a problem that anti-virus companies face all the time. In this paper we present detection algorithms that can help the antivirus community to ensure a variant of a known malware can still be detected without the need of creating a signature; a similarity analysis (based on specific quantitative measures) is performed to produce a matrix of similarity scores that can be utilized to determine the likelihood that a piece of code under inspection contains a particular malware. Two general malware detection methods presented in this paper are: Staassembly calls for analysis and SAVE uses API calls (Static API call sequence and Static API call set) for analysis. We show where Assembly can be superior to API calls in that it allows a more detailed comparison of executables. API calls, on the other hand, can be superior to Assembly for its speed and its smaller signature. Our two proposed techniques are implemented in SAVE) and MEDiC. We present experimental results that indicate that both of our proposed techniques can provide a better detection performance against obfuscated malware. We also found a few false positives, such as those programs that use network functions (e.g. PuTTY) and encrypted programs (no API calls or assembly functions are found in the source code) when the thresholds are set 50% similarity measure. However, these false positives can be minimized, for example by changing the threshold value to 70% that determines whether a program falls in the malicious category or not.

A Novel Approach to Detect Malware Based on API Call Sequence Analysis

“…Others extract API call sequence for each class and develop static signature based on it [8][9][10][11]. They are better from the semantic view because they monitor the sequence of calls and the flow of programs.…”

“…With static approach [6,10,11,21], API list can be extracted from PE format of the executable files. With dynamic approach [7][8][9][22][23][24], the APIs that are called can be observed by running the executable files (usually run on virtual machine). There are two major ways to analyze the API call information gathered through the static approach.…”

Graph-based malware detection using dynamic analysis

“…These data sources could include information based on the static analysis of the binary and the API sequence calls made by the program. Methods based on these data sources have been shown to be successful [30,33,39], and could possibly lead to more accurate results when combined in our multiple kernel learning framework.…”

Research on Malware Classification with Network Activity for Classification and Attack Prediction of Attack Groups

“…Shankarapani et al [6] presented a methodology for composing signatures of malicious codes from PEs for identifying known and unknown malware. The key assumption of their idea is that to preserve its functionality a polymorphic malware should contain a sufficiently similar API calling sequence or assembly code.…”

“…The authors later enhanced it and presented a sequence alignment method using binary scoring and a signature-updating scheme to detect masquerade attacks [20] . Another recent approach to detection is analyzing API call sequences and classifying them as benign or malicious [21] .…”