Keeping pace with the creation of new malicious PDF files using an active-learning based detection framework

Abstract

Attackers increasingly take advantage of naive users who tend to treat non-executable files casually, as if they are benign. Such users often open non-executable files although they can conceal and perform malicious operations. Existing defensive solutions currently used by organizations prevent executable files from entering organizational networks via web browsers or email messages. Therefore, recent advanced persistent threat attacks tend to leverage non-executable files such as portable document format (PDF) documents which are used daily by organizations. Machine Learning (ML) methods have recently been applied to detect malicious PDF files, however these techniques lack an essential element-they cannot be efficiently updated daily. In this study we present an active learning (AL) based framework, specifically designed to efficiently assist anti-virus vendors focus their analytical efforts aimed at acquiring novel malicious content. This focus is accomplished by identifying and acquiring both new PDF files that are most likely malicious and informative benign PDF documents. These files are used for retraining and enhancing the knowledge stores of both the detection model and anti-virus. We propose two AL based methods: exploitation and combination. Our methods are evaluated and compared to existing AL method (SVM-margin) and to random sampling for 10 days, and results indicate that on the last day of the experiment, combination outperformed all of the other methods, enriching the signature repository of the anti-virus with almost seven times more new malicious PDF files, while each day improving the detection model's capabilities further. At the same time, it dramatically reduces security experts' efforts by 75 %. Despite this significant reduction, results also indicate that our framework better detects new malicious PDF files than leading anti-virus tools commonly used by organizations for protection against malicious PDF files.

PDF-Malware Detection: A Survey and Taxonomy of Current Techniques

“…Another commonly used string obfuscation technique is the application of de-obfuscation functions upon strings at runtime, like substitution or XOR. Obfuscated code can be placed in any object and then deobfuscated only at runtime [13]. This kind of approach is extremely powerful against static analysis, while it is potentially subject to detection with dynamic analysis approaches.…”

“…ShellOS kernel, runs as a guest OS using Kernel-based Virtual MAchine (KVM). It communicates with the host operating system by mean of shared memory address space regions, through witch it receives the stream of code to analyze and writes back the results Active Learning Framework -Nissim et al [13] proposed an Active Learning (AL) based framework, specifically designed to efficiently assist anti-virus vendors focussing their analytical efforts aimed at acquiring novel malicious content. The objective is to identify and acquire both new PDF files that are most likely malicious and informative benign PDF documents.…”

Inter-labeler and intra-labeler variability of condition severity classification models using active and passive learning methods

“…This selection is expected to decrease the number of conditions that experts need to manually review and label. Studies in several domains have successfully applied AL to reduce the resources (i.e., time and money) required for labeling examples (25, 26, 27,68,69,70,71,81,82, 83). AL is divided roughly into two major approaches: 1) membership queries (28) in which examples are artificially generated from the problem space; and 2) selective-sampling (29) in which examples are selected from a pool, which is the focus of this paper.…”

MalJPEG: Machine Learning Based Solution for the Detection of Malicious JPEG Images

“…In August 2017, it was reported that SyncCrypt ransomware was spread using JPEG images. 7 In December 2018, Trend Micro, 8 an enterprise cyber security company, reported that cyber criminals used memes on Twitter(JPEG images) in order to convey commands to malware. 9 Recently, in December 2019, researchers from the Sophos security company published a comprehensive report 10 on the MyKings cryptomining botnet that lurks behind a seemingly innocuous JPEG of Taylor Swift.…”

“…When a new malware, or new variant of existing malware appears, there is a time lag until these defense solutions update their clients with the new signature-a time in which the clients are vulnerable to the new malware. In contrast, in recent years, machine learning (ML) algorithms have demonstrated their ability to detect both known and unknown malware in various domains, particularly for the detection of malware in various types of files [1]- [7]. However, to the best of our knowledge, machine learning methods have not been employed for the detection of malicious JPEG images.…”