Behavioral clustering of non-stationary IP flow record data

Christian, Hammerschmidt; Marchal, Samuel; State, Radu; Sicco, Verwer

doi:10.1109/cnsm.2016.7818436

Cited by 11 publications

(13 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Discretizing sequences is typically faster and makes measuring distances easier. Pellegrino et al [43] learn state machines from discretized netflow data in order to detect bot-infected traffic, while Hammerschmidt et al [27] use it to cluster host behavior over time. Lin et al [36] detect anomalies in industrial water treatment plant by using discretized sequences from sensor readings.…”

Section: Challenges In Malware Behavior Modelingmentioning

confidence: 99%

Beyond Labeling: Using Clustering to Build Network Behavioral Profiles of Malware Families

Nadeem

Hammerschmidt

Gañán

et al. 2020

Malware Analysis Using Artificial Intelligence and Deep Learning

Self Cite

View full text Add to dashboard Cite

Malware family labels are known to be inconsistent. They are also blackbox since they do not represent the capabilities of malware. The current state of the art in malware capability assessment includes mostly manual approaches, which are infeasible due to the ever-increasing volume of discovered malware samples. We propose a novel unsupervised machine learning-based method called MalPaCA, which automates capability assessment by clustering the temporal behavior in malware's network traces. MalPaCA provides meaningful behavioral clusters using only 20 packet headers. Behavioral profiles are generated based on the cluster membership of malware's network traces. A Directed Acyclic Graph shows the relationship between malwares according to their overlapping behaviors. The behavioral profiles together with the DAG provide more insightful characterization of malware than current family designations. We also propose a visualization-based evaluation method for the obtained clusters to assist practitioners in understanding the clustering results. We apply MalPaCA on a financial malware dataset collected in the wild that comprises 1.1 k malware samples resulting in 3.6 M packets. Our experiments show that (i) MalPaCA successfully identifies capabilities, such as port scans and reuse of Command and Control servers; (ii) It uncovers multiple discrepancies between behavioral clusters and malware family labels; and (iii) It demonstrates the effectiveness of clustering traces using temporal features by producing an error rate of 8.3%, compared to 57.5% obtained from statistical features.

show abstract

Section: Challenges In Malware Behavior Modelingmentioning

confidence: 99%

Beyond Labeling: Using Clustering to Build Network Behavioral Profiles of Malware Families

Nadeem

Hammerschmidt

Gañán

et al. 2020

Malware Analysis Using Artificial Intelligence and Deep Learning

Self Cite

View full text Add to dashboard Cite

show abstract

“…The majority of the previous works focus on the profiling network behavior of individual hosts [3,10,26], their classification [14,16] and clustering [21]. The authors of [10] use the change point detection techniques and the indicator of "freshness" to cluster the hosts according to its different activities over time. IP flow records are used as a data source, and the authors test their approach to modeling the different behavior of botnet hosts from real-world data observation.…”

Section: Related Workmentioning

confidence: 99%

Behavior-Aware Network Segmentation using IP Flows

Smeriga

Jirsík

2019

Proceedings of the 14th International Conference on Availability, Reliability and Security

View full text Add to dashboard Cite

Network segmentation is a powerful tool for network defense. In contemporary complex, dynamic, and multilayer networks, network segmentation suffers from lack of visibility into processes in the network, which results in less strict segment definition and loosen network security. Moreover, the dynamics of the networks makes the manual identification of network segments nearly impossible. In this paper, we inspect the possibilities of the behavioraware network segmentation using IP flows and machine learning approaches that would enable to identify segments automatically, even in a complex network. We evaluate the suitability of clustering algorithms for identification of behavior-consistent segments in a network. We show that the clustering algorithms can identify relevant behavior-consistent clusters that overlap with those identified manually by experts. Apart from the segment identification, we investigate the other essential task of network segmentation process: assignment of an unknown host to an existing segment. We evaluate the performance of four different classification mechanisms on a real-world dataset. We show that it is possible to assign an unknown host to an appropriate network segment with up to 92% precision. Moreover, we release the whole dataset and experiment steps available for public use.

show abstract

“…Nevertheless, the current state-of-the-art host profiling research rarely takes the real-world temporal aspects into account. The datasets utilized for profile creation and temporal evaluation span from 15 minutes in [7] to one month period in [8]. The length of these time spans is insufficient to reflect the long-term behavior of a host reliably.…”

Section: Introductionmentioning

confidence: 99%

Host Behavior in Computer Network: One-Year Study

Jirsík

Velan

2021

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

An analysis of a host behavior is an essential key for modern network management and security. A robust behavior profile enables the network managers to detect anomalies with high accuracy, predict the host behavior, or group host to clusters for better management. Hence, host profiling methods attract the interest of many researchers, and novel methods for host profiling are being introduced. However, these methods are frequently developed on preprocessed and small datasets. Therefore, they do not reflect the real-world artifacts of the host profiling, such as missing observations, temporal patterns, or variability in the profile characteristics in time. To provide the needed insight into the artifacts of host profiling in real-world settings, we present a study of the host behavior in a network conducted on a oneyear-long real-world network dataset. In the study, we inspect the availability of the data for host profiling, identify the temporal patterns in host behavior, introduce a method for stable labeling of the hosts, and assess the variability of the host characteristics in the course of the year using the coefficient of variance. Moreover, we make the one-year dataset containing nine characteristics used for host behavior analysis available for public use and further research, including selected use cases representing host profiling caveats. We also share the record of analyses presented in the paper.

show abstract

Behavioral clustering of non-stationary IP flow record data

Cited by 11 publications

References 18 publications

Beyond Labeling: Using Clustering to Build Network Behavioral Profiles of Malware Families

Beyond Labeling: Using Clustering to Build Network Behavioral Profiles of Malware Families

Behavior-Aware Network Segmentation using IP Flows

Host Behavior in Computer Network: One-Year Study

Contact Info

Product

Resources

About