Collecting and characterizing a real broadband access network traffic dataset

Lopez, Martin Andreoni; Silva, Renato S.; Alvarenga, Igor Drummond; Rebello, Gabriel Antonio F.; Sanz, Igor Jochem; Lobato, Antonio Gonzalez Pastana; Mattos, Diogo M. F.; Duarte, Otto Carlos Muniz Bandeira; Pujolle, Guy

doi:10.1109/csnet.2017.8241999

Cited by 18 publications

(18 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Network traffic is abstracted in 26 features and contains three classes, DoS, probe, and normal traffic. The third dataset is the NetOp, a real dataset from a Brazilian operator . The dataset contains anonymized access traffic of 373 broadband users of the South Zone of the city of Rio de Janeiro.…”

Section: Catraca Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Toward a monitoring and threat detection system based on stream processing as a virtual network function for big data

Lopez

Mattos

Duarte

et al. 2019

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

The late detection of security threats causes a significant increase in the risk of irreparable damages and restricts any defense attempt. In this paper, we propose a sCAlable TRAffic Classifier and Analyzer (CATRACA). CATRACA works as an efficient online Intrusion Detection and Prevention System implemented as a Virtualized Network Function. CATRACA is based on Apache Spark, a Big Data Streaming processing system, and it is deployed over the Open Platform for Network Functions Virtualization (OPNFV), providing an accurate real-time threat-detection service. The system presents a friendly graphical interface that provides real-time visualization of the traffic and the attacks that occur in the network. Our prototype can differentiate normal traffic from denial of service (DoS) attacks and vulnerability probes over 95% accuracy under three different datasets. Moreover, CATRACA handles streaming data under concept drift detection with more than 85% of accuracy. KEYWORDSbig data, network traffic classification, stream processing, threat detection, virtual network function INTRODUCTIONThe Internet is facing constant changes, from the diversity of the user, the complexity of its application, until the heterogeneity of the information producers. 1 As a consequence, traffic monitoring, a critical task in maintaining the stability, reliability, and security of computer networks, is facing new challenges. 2 Current network monitoring tools are inadequate for current speed and management needs of large network domains.To ensure network security, new systems must be designed since current security systems such as Security Information and Event Management (SIEM) are inadequate. While 82% of security threats occur in minutes, an intrusion can take up to 8 months to be detected. 3 It is essential that the detection time is the least possible so that intrusion prevention can be effective. 4Security incidents have increased their complexity, and simple analysis and filtering of packets are no longer sufficient. Attackers try to hide malicious traffic from the security tools by forging the source IP and dynamically changing TCP port. In this context, a promising alternative for classifying network traffic and detect threats is to apply Machine Learning (ML) techniques. These techniques are suitable for big data, with more samples to train the classifier, as methods have higher effectiveness. 5 With a large number of features, however, ML techniques perform results with high latency due to computational resource consumption. This high latency is a disadvantage for applications that use machine learning for real-time classification. For example, network monitoring applications must analyze data and detect threats as quickly as possible. In this context, real-time stream processing allows the immediate analysis of different types of data and consequently benefits traffic monitoring for security threat detection. Open source distributed processing platforms such as Apache Storm, 6 Apache Flink, 7 and Apache Spark 8 process big data w...

show abstract

Section: Catraca Evaluationmentioning

confidence: 99%

“…With big dataset, a computer cluster must be used to introduce low latency when processing the data. A previous work analyzed just one day of the network traffic. In this paper, we analyze the entire dataset, in total of one full week.…”

Section: Catraca Evaluationmentioning

confidence: 99%

Toward a monitoring and threat detection system based on stream processing as a virtual network function for big data

Lopez

Mattos

Duarte

et al. 2019

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

show abstract

“…The attack data were obtained from the data collected by Garcia et al in a study on the behavior of botnets [24]. The dataset consists of flows identified by a tuple composed of the source and destination IP address, source and destination transport ports, and transport protocol [25]. Our compiled dataset counts with traffic from 15 devices both from captured laboratory usage traffic and the botnet dataset.…”

Section: Virtual Network Function Performancementioning

confidence: 99%

“…We added 10 new features into the dataset, that marks if the flow belongs to the specific service. It is important to use tagging features to keep some information about the service whereas we still are able to calculate correlation and to apply the principal components analysis over the dataset [25]. Figure 5a compares the accuracy of three classification algorithms: probabilistic neural network (PNN); multilayer perceptron neural network (MLP) and Boosted Decision Trees [26].…”

Section: Virtual Network Function Performancementioning

confidence: 99%

An agile and effective network function virtualization infrastructure for the Internet of Things

Mattos

Velloso

Duarte

2019

J Internet Serv Appl

View full text Add to dashboard Cite

The processing and power-consumption constraints of the Internet of Things devices hinder them to offer more complex network services than the simple data transmission in smart city scenarios. The lack of complex services, such as security and quality of service, can even foster disasters in urban centers. In this paper, we propose the integration of complex network services from the IoT devices till a cloud environment through an agile and effective network function virtualization infrastructure of isolated IoT domains. Therefore, our proposal develops a simple gateway access node that virtualizes the domains to which the devices connect. A prototype for services of security and quality of service has been implemented and its evaluation shows that virtualization of the access node does not impact the performance of virtual network functions. The results also show that the proposal provides security for IoT devices, identifying malicious traffic with 99.8% accuracy, avoiding denial of essential services, and ensuring the quality of service.

show abstract

“…Os perfis podem apontar usos característicos de determinadas aplicações [Shye et al, 2010] e o quanto de recursosé consumido por cada aplicação [Qian et al, 2011]. Contudo, em trabalhos anteriores a busca por padrões de uso de aplicações ou padrões de uso de recursos de rede são realizadas de forma supervisionada, istoé, conjuntos de dados característicos de cada aplicação são usados para treinar um classificador capaz de reconhecer os padrões em um conjunto de dados de teste [Andreoni Lopez et al, 2017]. Ao identificar padrões anteriormente ocultos de uso da redeé possível extrair não somente conhecimento a respeito do funcionamento da rede [Biswas et al, 2015, Ghosh et al, 2011, mas também a respeito dos usuários, como preferências e padrões de mobilidade [Wang et al, 2014, Guo et al, 2014.…”

Section: A Inferência De Perfis De Uso Da Redeunclassified

Uma Abordagem Não Supervisionada para Inferir Qualidade de Experiência em Redes Sem Fio de Grande Escala

Mattos¹,

Medeiros²,

Fernandes³

et al. 2019

Anais Do Workshop De Gerência E Operação De Redes E Serviços

View full text Add to dashboard Cite

Inferir a qualidade da experiência de usuários de redes sem fio é desafiador, pois o monitoramento da rede não captura a qualidade para cada usuário individualmente. Este artigo propõe uma abordagem não supervisionada, baseada em aprendizado de máquina, para inferir a qualidade de experiência de diferentes perfis de uso de uma rede sem fio de grande escala. A abordagem proposta usa a correlação entre dados de uso de pontos de acesso e estatísticas de fluxos de dados na rede. A ideia central da proposta é coletar dados de utilização de diversos pontos de acesso, correlacioná-los com as estatísticas dos fluxos das conexões que passam pelos pontos de acesso, reportados pelo NetFlow, e, a partir da aplicação do algoritmo de agrupamento k-means, inferir diferentes perfis de uso da rede. A abordagem proposta foi avaliada na rede sem fio real de grande escala e os resultados mostram que a separação dos fluxos em cinco agrupamentos permite identificar perfis característicos de estados degradados da rede e situações de sobrecarga em pontos de acesso, considerando apenas as estatísticas de fluxos reportadas.

show abstract

Collecting and characterizing a real broadband access network traffic dataset

Cited by 18 publications

References 18 publications

Toward a monitoring and threat detection system based on stream processing as a virtual network function for big data

Toward a monitoring and threat detection system based on stream processing as a virtual network function for big data

An agile and effective network function virtualization infrastructure for the Internet of Things

Uma Abordagem Não Supervisionada para Inferir Qualidade de Experiência em Redes Sem Fio de Grande Escala

Contact Info

Product

Resources

About