A Survey on Big Data for Network Traffic Monitoring and Analysis

D’Alconzo, Alessandro; Drago, Idílio; Morichetta, Andrea; Mellia, Marco; Casas, Pedro

doi:10.1109/tnsm.2019.2933358

Cited by 127 publications

(50 citation statements)

References 109 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…the connection to the C&C server might be persistent; and DNS features inspired from [24]. We also plan to develop a realtime implementation of our algorithm, working with sliding time windows and incrementally updating the model, based on [47]. 1 Observation of bots fingerprints Fig.…”

Section: Discussionmentioning

confidence: 99%

Botnet Fingerprinting: A Frequency Distributions Scheme for Lightweight Bot Detection

Blaise

Bouet

Conan

et al. 2020

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

Efficient bot detection is a crucial security matter and widely explored in the past years. Recent approaches supplant flow-based detection techniques and exploit graphbased features, incurring however in scalability issues, with high time and space complexity. Bots exhibit specific communication patterns: they use particular protocols, contact specific domains, hence can be identified by analyzing their communication with the outside. A way we follow to simplify the communication graph and avoid scalability issues is looking at frequency distributions of protocol attributes capturing the specificity of botnets behaviour. We propose a bot detection technique named BotFP, for BotFin-gerPrinting, which acts by (i) characterizing hosts behaviour with attribute frequency distribution signatures, (ii) learning benign hosts and bots behaviours through either clustering or supervised Machine Learning (ML), and (iii) classifying new hosts either as bots or benign ones, using distances to labelled clusters or relying on a ML algorithm. We validate BotFP on the CTU-13 dataset, which contains 13 scenarios of bot infections, connecting to a Command-and-Control (C&C) channel and launching malicious actions such as port scanning or Denial-of-Service (DDoS) attacks. Compared to state-of-the-art techniques, we show that BotFP is more lightweight, can handle large amounts of data, and shows better accuracy.

show abstract

Section: Discussionmentioning

confidence: 99%

Botnet Fingerprinting: A Frequency Distributions Scheme for Lightweight Bot Detection

Blaise

Bouet

Conan

et al. 2020

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

show abstract

“…The study of traffic analysis for network data flow is not new. Given the importance of traffic classification for purposes of security and quality of service (QoS) management, different machine learning solutions have been used effectively [2,[13][14][15][16][17].…”

Section: Literaturementioning

confidence: 99%

OPriv: Optimizing Privacy Protection for Network Traffic

Chaddad

Chehab

Kayssi

2021

JSAN

View full text Add to dashboard Cite

Statistical traffic analysis has absolutely exposed the privacy of supposedly secure network traffic, proving that encryption is not effective anymore. In this work, we present an optimal countermeasure to prevent an adversary from inferring users’ online activities, using traffic analysis. First, we formulate analytically a constrained optimization problem to maximize network traffic obfuscation while minimizing overhead costs. Then, we provide OPriv, a practical and efficient algorithm to solve dynamically the non-linear programming (NLP) problem, using Cplex optimization. Our heuristic algorithm selects target applications to mutate to and the corresponding packet length, and subsequently decreases the security risks of statistical traffic analysis attacks. Furthermore, we develop an analytical model to measure the obfuscation system’s resilience to traffic analysis attacks. We suggest information theoretic metrics for quantitative privacy measurement, using entropy. The full privacy protection of OPriv is assessed through our new metrics, and then through extensive simulations on real-world data traces. We show that our algorithm achieves strong privacy protection in terms of traffic flow information without impacting the network performance. We are able to reduce the accuracy of a classifier from 91.1% to 1.42% with only 0.17% padding overhead.

show abstract

“…This has scalability issues. Thus, recently proposed big data analytic systems-see [63], [64], [65], [66], [67] as well as [68] and references within-suggest to use a distributed setup whereby data is locally preprocessed, e.g., by aggregation or sampling, and then centrally analyzed. This reduces the need to transfer the raw data.…”

Section: State Of the Artmentioning

confidence: 99%

Exploring Network-Wide Flow Data With Flowyager

Saidi

Maghsoudlou

Foucard

et al. 2020

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

Many network operations, ranging from attack investigation and mitigation to traffic management, require answering network-wide flow queries in seconds. Although flow records are collected at each router, using available traffic capture utilities, querying the resulting datasets from hundreds of routers across sites and over time, remains a significant challenge due to the sheer traffic volume and distributed nature of flow records. In this paper, we investigate how to improve the response time for a priori unknown network-wide queries. We present Flowyager, a system that is built on top of existing traffic capture utilities. Flowyager generates and analyzes tree data structures, that we call Flowtrees, which are succinct summaries of the raw flow data available by capture utilities. Flowtrees are self-adjusted data structures that drastically reduce space and transfer requirements, by 75% to 95%, compared to raw flow records. Flowyager manages the storage and transfers of Flowtrees, supports Flowtree operators, and provides a structured query language for answering flow queries across sites and time periods. By deploying a Flowyager prototype at both a large Internet Exchange Point and a Tier-1 Internet Service Provider, we showcase its capabilities for networks with hundreds of router interfaces. Our results show that the query response time can be reduced by an order of magnitude when compared with alternative data analytics platforms. Thus, Flowyager enables interactive network-wide queries and offers unprecedented drill-down capabilities to, e.g., identify DDoS culprits, pinpoint the involved sites, and determine the length of the attack. 1 NetFlow is a Cisco trademark, so other vendors market the NetFlow support with other names, e.g., Juniper Networks use the trademark Jflow or cflowd. 2 NetFlow and IPFIX capabilities are available in router series, e.g., Cisco IOS-XR, IOS and Catalyst router [

show abstract

A Survey on Big Data for Network Traffic Monitoring and Analysis

Cited by 127 publications

References 109 publications

Botnet Fingerprinting: A Frequency Distributions Scheme for Lightweight Bot Detection

Botnet Fingerprinting: A Frequency Distributions Scheme for Lightweight Bot Detection

OPriv: Optimizing Privacy Protection for Network Traffic

Exploring Network-Wide Flow Data With Flowyager

Contact Info

Product

Resources

About