Classification of malicious process using high‐level activity based dynamic analysis

Abstract

Detection of malicious program is a critical component in system security. Advanced malware like metamorphic, polymorphic, or evasive malware cannot be detected successfully using the traditional signature‐ and heuristics‐based anti‐malware methods. This paper proposes an approach to detect malicious activity running on the system by leveraging semantics of the behavior of processes to complement the static characteristics of their executable image. Static characteristics are captured from the file header, optional headers, and sections of the executable file. The behavior model of the process is in the form of high‐level activities generated through the runtime tracing of the process. The proposed approach classifies malicious executables on the basis of the higher‐level abstraction of the activity rather than the actual system calls. Experimental results show that the approach supplements the traditional methods toward the improvement of the precision in the overall classification of malicious activity, besides detecting advanced malware which uses obfuscation, polymorphism techniques, etc. for evasion.