Intelligent SDN based traffic (de)Aggregation and Measurement Paradigm (iSTAMP)

Abstract

Fine-grained traffic flow measurement, which provides useful information for network management tasks and security analysis, can be challenging to obtain due to monitoring resource constraints. The alternate approach of inferring flow statistics from partial measurement data has to be robust against dynamic temporal/spatial fluctuations of network traffic. In this paper, we propose an intelligent Traffic (de)Aggregation and Measurement Paradigm (iSTAMP), which partitions TCAM entries of switches/routers into two parts to: 1) optimally aggregate part of incoming flows for aggregate measurements, and 2) de-aggregate and directly measure the most informative flows for per-flow measurements. iSTAMP then processes these aggregate and perflow measurements to effectively estimate network flows using a variety of optimization techniques. With the advent of Software-Defined-Networking (SDN), such real-time rule (re)configuration can be achieved via OpenFlow or other similar SDN APIs. We first show how to design the optimal aggregation matrix for minimizing the flow-size estimation error. Moreover, we propose a method for designing an efficient-compressive flow aggregation matrix under hard resource constraints of limited TCAM sizes. In addition, we propose an intelligent Multi-Armed Bandit based algorithm to adaptively sample the most "rewarding" flows, whose accurate measurements have the highest impact on the overall flow measurement and estimation performance. We evaluate the performance of iSTAMP using real traffic traces from a variety of network environments and by considering two applications: traffic matrix estimation and heavy hitter detection. Also, we have implemented a prototype of iSTAMP and demonstrated its feasibility and effectiveness in Mininet environment.

Towards accurate online traffic matrix estimation in software-defined networks

“…AL is given and it is fixed while AS is determined by the flow rules designed by the controller to provide the most informative aggregate measurements adhering to the routing policy. Having measurements YS and YL as well as aggregation matrices AS and AL, the traffic matrix X can be estimated using the following optimization formulation (2), which is a convex optimization problem that is effective for estimating highly fluctuating network flows [15].X = minimize rnew.priority ← r old .priority + 1 9:…”

“…However, how to find out the expected large flows is a problem. To solve this problem, iSTAMP [15] uses a two-phase approach, which measures the sizes of all the flows using available TCAM entries over multiple epochs in the first phase, and selects the k largest flows to measure in the second phase. However, measuring the per-flow sizes is costly and time consuming, especially when the available TCAM entries are limited and the number of flows is large.…”

“…OpenTM [13], DCM [14] and iSTAMP [15] aim to measure TM in SDN. OpenTM and DCM are per-flow based measurement solutions, which directly measure and estimate T-M by tracking statistics of each flow.…”

Dynamic Core VNT Adaptability Based on Predictive Metro-Flow Traffic Models

“…In this regard, some authors have proposed to monitor (small) individual flows instead of the (large) aggregated flows for traffic modelling purposes. For instance, authors in [9] proposed a machine learning procedure that intelligently deaggregates relevant monitored traffic flows and aggregates the rest to achieve accurate traffic estimations. However, this kind of selective flow monitoring approaches lacks the flexibility required to adapt OD traffic models against any potential metroflow rerouting.…”

Parallelizing Under-Determined Inverse Problems for Network Applications

“…Due to limited measurement resources (e.g. flowtable/TCAM entries, storage capacity, and processing power) in network monitoring infrastructures, it is often infeasible and/or inefficient to directly measure the size of every flow (using NetFlow or sFlow) in large-scale networks [15]. In this case, a TME problem is formulated as an UDLI problem where the number of known SNMP link-load and/or flow aggregated measurements are less than the number of unknown ODFs [8].…”