Leaving All Proxy Server Logs to Paragraph Vector

Abstract

Cyberattack techniques continue to evolve every day. Detecting unseen drive-by-download attacks or C&C traffic is a challenging task. Pattern-matching-based techniques and using malicious blacklists are not efficient anymore, because attackers easily change the traffic pattern or infrastructure to avoid detection. Therefore, many behaviorbased detection methods have been proposed, which use the immutable characteristic of the traffic. These previous methods, however, focus on the attack technique, and can only detect drive-by-download (DbD) attacks or C&C traffic which have the immutable characteristic. These traditional methods have to devise the feature vectors. This paper proposes a generic detection method, which is independent of attack methods and does not need devising feature vectors. Our method uses Paragraph Vector, an unsupervised algorithm that learns fixed-length feature representations from variable-length texts and classifiers. Our method uses Paragraph Vector to capture the context in proxy server logs. We conducted cross-validation, timeline analysis and cross-dataset validation with multiple datasets. The experimental results show our method can detect unseen DbD attacks and C&C traffic in proxy server logs. The best F-measure achieved 0.97 in the timeline analysis and 0.96 on the other dataset.

An Attempt to Read Network Traffic with Doc2vec

“…In terms of accuracy, our method has slightly lower accuracy than the previous method [5]. In regard to other previous methods, it is not feasible to compare under fair conditions since many previous works use their own datasets which are not open to the public.…”

“…The alternative approach is classification based on network logs such as DNS records, NetFlow or proxy server logs. There are several methods which use NetFlow [2], [3], DNS records [13], [14], [15], [16] and proxy server logs [4], [5], [17], [18], [19], [20], [21], [22], [23], [24]. Some approaches focus on NN to detect basic network attacks [25], [26], [27].…”

“…To address these problems, neural networks can be applied to perform feature learning. Doc2vec is one of these models that learn fixed-length feature representation from variablelength documents and has been applied to proxy logs [5]. However, some attackers still use protocols other than http or https.…”

“…In this paper, we extend the previous method [5] to a generic detection method which supports any protocols. The key idea of this research is reading network packets as a natural language.…”

“…The main contributions of this paper are as follows: ( 1 ) Extends the previous method [5] to a generic detection method which supports any protocols. ( 2 ) Effectively uses Doc2vec in an application layer to classify traffic [7].…”