The Time-Varying Dependency Patterns of NetFlow Statistics

Abstract

We investigate where and how key dependency structure between NetFlow features change throughout the course of daily network activity. Our approach is probabilistic in nature, we formulate the identification of dependency patterns as a regularised statistical estimation problem. The resulting model can be interpreted as a set of time-varying graphs and provides a useful visual interpretation of network activity. We believe this is the first application of dynamic graphical modelling to network traffic of this kind. Investigations are performed on 9 days of realworld network traffic across a subset of IP's. We demonstrate that dependency between features may change across time and discuss how these change at an intra and inter-day level.

Adaptive regularization for Lasso models in the context of nonstationary data streams

“…Another exciting avenue would be to use the proposed methods to understand variability in dynamic functional connectivity [Monti et al, 2015]. Furthermore, it would also be interesting to consider alternative applications such as cyber-security [Gibberd et al, 2016], gene expression data [Gibberd and Nelson, 2017] and finance.…”

Characterising Dependency in Computer Networks using Spectral Coherence

“…There are a great variety of measures and methods that can be used to analyse dependency between streams, for instance through measures such as covariance [3], correlation [2], partial correlation [4], or higher-order measures such as cross-cumulants [5]. A traditional approach to network traffic modelling is to assume it is generated according to a Poisson point process [1,6].…”