A Neural Network-Based Approach for Cryptographic Function Detection in Malware

Abstract

Cryptographic technology has been commonly used in malware for hiding their static characteristics and malicious behaviors to avoid the detection of anti-virus engines and counter the reverse analysis from security researchers. The detection of cryptographic functions in an effective way in malware has vital significance for malicious code detection and deep analysis. Many efforts have been made to solve this issue, while existing methods suffer from some issues, such as unable to achieve promising results in accuracy, limited by prior knowledge, and have a high overhead. In this paper, we draw on the idea of text classification in the field of natural language processing and propose a novel neural network to detect the type of cryptographic functions. The new network is an end-2-end model which includes two important modules: Instruction-2-vec and K-Max-CNN-Attention. The Instruction-2-vec model extracts the ''words'' of assembly instructions and transfers them into continuous vectors. The K-Max-CNN-Attention is used to encode the instruction vectors and generate the representation of the function. And we designed a softmax classifier to predict the categories of the functions. Extensive experiments were conducted on a collected dataset which contains 15 common types of cryptographic functions extracted from malware, to assess the validity of the proposed approach. The experiment results showed that the proposed approach archives a better performance than the recent embedding network SAFE with the Precision, Recall and F1-score of 0.9349, 0.8933 and 0.9020, respectively. We also compared it with four widely-used tools, the results demonstrated that our approach is much better in accuracy and effectiveness than all of them.

Revisiting Binary Code Similarity Analysis Using Interpretable Feature Engineering and Lessons Learned

“…However, one may extend the scope to handle other BCSA scenarios to compare binaries [20], [27], [54] or a series of instructions [32], [34], [57]. Additionally, one can extend our approach for various purposes, such as vulnerability discovery [11], [12], [20], [23], [28], [59], [138], malware detection [5], [6], [139], [140], [141], [142], [143], library function identification [71], [84], [144], [145], [146], [147], plagiarism/authorship detection [8], [82], [148], or patch identification [149], [150], [151]. However, extending our work to other BCSA tasks may not be directly applicable.…”

The Dichotomy of Neural Networks and Cryptography: War and Peace

“…Cryptography is a tool that can protect the confidentiality of data, while often used to the benefit of users it can be used by attackers to hide malware on a system until it is needed or obfuscate network traffic. It can be difficult to identify malware that is able to encrypt data as it might require reverse engineering the suspicious program and applying a thorough understanding of many encryption methods [37]. It has been found that NNs could simplify the process of detecting encryption in obfuscated programs.…”

“…Jia et al [37] propose solving this issue with a NN model called K-max-CNN-Attention that looks for common instruction patterns rather than relative instruction density. The improvements this model brings is in the move to a convolutional NN (CNN), which interpret blocks of data maintaining the original structure and a better preprocessing scheme which simplifies the input enough to be meaningful to the NN but leaving more information to be interpreted, while these changes improve on performance and accuracy of existing techniques Jia et al [37] speculate that better accuracy could be achieved by changing the preprocessing and classification models to consider non-sequential execution of code.…”