Sensing the Noise: Uncovering Communities in Darknet Traffic

Soro, Francesca; Allegretta, Mauro; Mellia, Marco; Drago, Idílio; Bertholdo, Leandro Márcio

doi:10.1109/medcomnet49392.2020.9191555

Cited by 13 publications

(8 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a nutshell, the algorithm evaluates how much more densely connected the vertices within a cluster are when compared to how connected they would be in a random network with the same degree distribution. The Louvain algorithm has been successfully used for cluster detection in social networks [33] and even for darknet traffic analysis [39]. Among its advantages, the algorithm does not require a pre-defined number of clusters.…”

Section: Clustering Methodologymentioning

confidence: 99%

See 1 more Smart Citation

DarkVec

Gioacchini

Vassio

Mellia

et al. 2021

Proceedings of the 17th International Conference on Emerging Networking EXperiments and Technologies

Self Cite

View full text Add to dashboard Cite

Darknets are passive probes listening to traffic reaching IP addresses that host no services. Traffic reaching them is unsolicited by nature and often induced by scanners, malicious senders and misconfigured hosts. Its peculiar nature makes it a valuable source of information to learn about malicious activities. However, the massive amount of packets and sources that reach darknets makes it hard to extract meaningful insights. In particular, multiple senders contact the darknet while performing similar and coordinated tasks, which are often commanded by common controllers (botnets, crawlers, etc.). How to automatically identify and group those senders that share similar behaviors remains an open problem.We here introduce DarkVec, a methodology to identify clusters of senders (i.e., IP addresses) engaged in similar activities on darknets. DarkVec leverages word embedding techniques (e.g., Word2Vec) to capture the co-occurrence patterns of sources hitting the darknets. We extensively test DarkVec and explore its design space in a case study using one month of darknet data. We show that with a proper definition of service, the generated embeddings can be easily used to (i) associate unknown senders' IP addresses to the correct known labels (more than 96% accuracy), and (ii) identify new attack and scan groups of previously unknown senders. We contribute DarkVec source code and datasets to the community also to stimulate the use of word embeddings to automatically learn patterns on generic traffic traces.

show abstract

Section: Clustering Methodologymentioning

confidence: 99%

“…Authors of [39] build a bipartite graph for representing darknet traffic and then apply community detection on it, obtaining clusters of autonomous systems characterized by similar behavior. These approaches are complementary to DarkVec, as they focus on particular features of the traffic.…”

Section: Related Workmentioning

confidence: 99%

DarkVec

Gioacchini

Vassio

Mellia

et al. 2021

Proceedings of the 17th International Conference on Emerging Networking EXperiments and Technologies

Self Cite

View full text Add to dashboard Cite

show abstract

“…This methodology is based on (i) exploring different types of user interactions (e.g., users who used the same hashtag sequence or who retweeted the same tweet sequence) and (ii) applying a threshold-based approach to remove edges whose weights fall below a certain threshold. This simple threshold-based backbone extraction approach has been widely used in the literature [69][70][71][72][73][74], including studies on online hate communities [12] and communities with online news exposure [10]. However, some studies point to possible misinterpretation of results when using such approach, as they may introduce bias into the analysis [72].…”

Section: Prior Studies On Network Backbone Extractionmentioning

confidence: 99%

On network backbone extraction for modeling online collective behavior

et al. 2022

Self Cite

View full text Add to dashboard Cite

Collective user behavior in social media applications often drives several important online and offline phenomena linked to the spread of opinions and information. Several studies have focused on the analysis of such phenomena using networks to model user interactions, represented by edges. However, only a fraction of edges contribute to the actual investigation. Even worse, the often large number of non-relevant edges may obfuscate the salient interactions, blurring the underlying structures and user communities that capture the collective behavior patterns driving the target phenomenon. To solve this issue, researchers have proposed several network backbone extraction techniques to obtain a reduced and representative version of the network that better explains the phenomenon of interest. Each technique has its specific assumptions and procedure to extract the backbone. However, the literature lacks a clear methodology to highlight such assumptions, discuss how they affect the choice of a method and offer validation strategies in scenarios where no ground truth exists. In this work, we fill this gap by proposing a principled methodology for comparing and selecting the most appropriate backbone extraction method given a phenomenon of interest. We characterize ten state-of-the-art techniques in terms of their assumptions, requirements, and other aspects that one must consider to apply them in practice. We present four steps to apply, evaluate and select the best method(s) to a given target phenomenon. We validate our approach using two case studies with different requirements: online discussions on Instagram and coordinated behavior in WhatsApp groups. We show that each method can produce very different backbones, underlying that the choice of an adequate method is of utmost importance to reveal valuable knowledge about the particular phenomenon under investigation.

show abstract

“…The address space which is not used on the internet is called darknets or network telescopes. Darknet traffic is not speculated over the internet to interact with other computers and only passively accepts incoming packets without generating outgoing packets [5]. Tor is a virtual computer network, allows users to gain access to hidden Darknet resources.…”

Section: Introductionmentioning

confidence: 99%

“…To achieve the detection objective, authors used ML, and Deep Learning (DL) techniques [15], [16], [17], [18]. Authors in [5], [19], [20] explored whether graph mining techniques can help to uncover such macroscopic coordinated events in darknet traffic. The darknet traffic is getting complex daily, and malicious activities such as physical threats, sales data theft, fraudulent activity, phishing attacks, and scams, DDoS attacks, illicit links, Illicit Drugs, and terrorism vary day by day [21], [22], [23], [24].…”

Section: Introductionmentioning

confidence: 99%

DarkDetect: Darknet Traffic Detection and Categorization Using Modified Convolution-Long Short-Term Memory

et al. 2021

View full text Add to dashboard Cite

Darknet is commonly known as the epicenter of illegal online activities. An analysis of darknet traffic is essential to monitor real-time applications and activities running over the Darknet. Recognizing network traffic bound to unused Internet addresses has become undeniably significant for identifying and examining malicious activities on the internet. Since there are no authentic hosts or devices in an unused address block, any observed network traffic must be the aftereffect of misconfiguration from spoofed source addressed and other frameworks that monitor unused address space. However, the recent advancements in artificial intelligence allow digital systems to detect and identify darknet traffic autonomously. In this paper, we propose a generalized approach for darknet traffic detection and categorization using Deep Learning. We examine the state-of-the-art complex dataset, which provides excessive information about the darknet traffic and perform data preprocessing. Next, we analyze diverse feature selection techniques to select optimal features for darknet traffic detection and categorization. We apply fine-tuned machine learning (ML) algorithms which include Decision Tree (DT), Gradient Boosting (GB), Random Forest Regressor (RFR), and Extreme Gradient Boosting (XGB) on selected features and compare the performance. Next, we apply modified Convolution-Long Short-Term Memory (CNN-LSTM) and Convolution-Gradient Recurrent Unit (CNN-GRU) deep learning techniques to recognize the network traffic more accurately. The results demonstrate that the proposed approach outperforms the existing approaches by yielding the maximum accuracy of 96% of darknet traffic detection and 89% of darknet traffic categorization through XGB as a feature selection approach and CNN-LSTM a recognition model.

show abstract

Sensing the Noise: Uncovering Communities in Darknet Traffic

Cited by 13 publications

References 26 publications

DarkVec

DarkVec

On network backbone extraction for modeling online collective behavior

DarkDetect: Darknet Traffic Detection and Categorization Using Modified Convolution-Long Short-Term Memory

Contact Info

Product

Resources

About