eXpose: A Character-Level Convolutional Neural Network with Embeddings For Detecting Malicious URLs, File Paths and Registry Keys

Abstract

For years security machine learning research has promised to obviate the need for signature based detection by automatically learning to detect indicators of attack. Unfortunately, this vision hasn't come to fruition: in fact, developing and maintaining today's security machine learning systems can require engineering resources that are comparable to that of signature-based detection systems, due in part to the need to develop and continuously tune the "features" these machine learning systems look at as attacks evolve. Deep learning, a subfield of machine learning, promises to change this by operating on raw input signals and automating the process of feature design and extraction. In this paper we propose the eXpose neural network, which uses a deep learning approach we have developed to take generic, raw short character strings as input (a common case for security inputs, which include artifacts like potentially malicious URLs, file paths, named pipes, named mutexes, and registry keys), and learns to simultaneously extract features and classify using character-level embeddings and convolutional neural network. In addition to completely automating the feature design and extraction process, eXpose outperforms manual feature extraction based baselines on all of the intrusion detection problems we tested it on, yielding a 5%-10% detection rate gain at 0.1% false positive rate compared to these baselines.

First Step Towards EXPLAINable DGA Multiclass Classification

“…eXpose. Saxe and Berlin [22] proposed eXpose, a CNN-based classifier for detecting malicious URLs, file paths and registry keys. The main difference of this model compared to NYU lies in the usage of the CNN layers.…”

“…On the other side, contextless approaches (e.g. [6,22,24,29,31]) entirely rely on information extracted from a single domain name for classification and are thus less resource intensive and less privacy invasive. Prior studies (e.g.…”

“…[24]), and deep learning based approaches, such as recurrent (RNNs), convolutional (CNNs), or residual neural networks (ResNets) (e.g. [6,22,29,31]). For the multiclass task, however, only deep learning based contextless classifiers are available to date.…”

The More, the Better? A Study on Collaborative Machine Learning for DGA Detection

“…Various approaches have been proposed in the past to capture DGA activity within networks. These approaches can be roughly divided into context-less (e.g., [10,43,45,56,61]) and context-aware approaches (e.g., [3,6,19,44,47,58]). The former group uses information extracted only from individual domain names and ignores any contextual data to separate benign from DGA-generated domains.…”

“…The proposed context-less ML classifiers can be further divided into feature-based such as support vector machines or random forests (e.g., [45]), and feature-less (DL) classifiers such as recurrent (RNN), convolutional (CNN), or residual neural networks (ResNet) (e.g., [10,43,56,61]). When comparing both types of classifiers, it was shown that the approaches based on DL achieve superior detection performance [10,37,51,56].…”