Threat Miner - A Text Analysis Engine for Threat Identification Using Dark Web Data

Abstract

Cyber threats continue to grow with novel methods to attack computing systems, highlighting the need for sophisticated mechanisms and techniques to protect against such dynamic threats. Contemporary cyber defence mechanisms utilise a range of methods which rely on monitoring network or system-level events. However, with the growing use of the dark web by mal-actors to share exploits, breaches, and data leaks, the use of such information to strengthen defence mechanisms becomes an intriguing prospect. In this paper, we present our efforts to develop a text mining engine (Threat Miner) which analyses data from dark web forums and transforms it into actionable intelligence. Leveraging cutting-edge machine learning techniques and utilising a bespoke threat dictionary, Threat Miner extracts useful information from dark web forums into STIX form, enabling it to be used with threat intelligence platforms. We also present the results of a thorough evaluation of our scheme which was conducted with the CrimeBB dataset [1] to understand the feasibility of the approach as well as its effectiveness in strengthening defence capability against cyber threats.

Dark Side of the Web: Dark Web Classification Based on TextCNN and Topic Modeling Weight

“…This study also employed TF-IDF for feature extraction and executed classification using Kmeans and DT [18]. Numerous analyses of the Dark Web have been conducted using word2vec [44], and there is active research on transforming extracted text features into matrices via embedding and implementing deep learning [45,46]. In the Dark Web context, text classification is predominantly utilized for preprocessing and feature extraction [16][17][18][40][41][42][43][44].…”