The problem of synthetically generating IP traffic matrices

Abstract

There exist a wide variety of network design problems that require a traffic matrix as input in order to carry out performance evaluation. The research community has not had at its disposal any information about how to construct realistic traffic matrices. We introduce here the two basic problems that need to be addressed to construct such matrices. The first is that of synthetically generating traffic volume levels that obey spatial and temporal patterns as observed in realistic traffic matrices. The second is that of assigning a set of numbers (representing traffic levels) to particular node pairs in a given topology. This paper provides an in-depth discussion of the many issues that arise when addressing these problems. Our approach to the first problem is to extract statistical characteristics for such traffic from real data collected inside two large IP backbones. We dispel the myth that uniform distributions can be used to randomly generate numbers for populating a traffic matrix. Instead, we show that the lognormal distribution is better for this purpose as it describes well the mean rates of origin-destination flows. We provide estimates for the mean and variance properties of the traffic matrix flows from our datasets. We explain the second problem and discuss the notion of a traffic matrix being well-matched to a topology. We provide two initial solutions to this problem, one using an ILP formulation that incorporates simple and well formed constraints. Our second solution is a heuristic one that incorporates more challenging constraints coming from carrier practices used to design and evolve topologies.

Towards a Statistical Characterization of the Interdomain Traffic Matrix

“…The tail of the distribution in Figure 1a can be modeled as Pareto (the CCDF in log-log scale resembles a straight line), while the distribution in Figure 1b can be modeled as LogNormal. This confirms previous observations of the heavy-tailed nature of sourced traffic distributions [1,6,11] with a more recent dataset. On the other hand, the distribution in Figure 1c decays faster than Pareto but slower than LogNormal.…”

“…The major reason for this, unfortunately, has been the lack of publicly available data [13] to enable such a study by the research community. An early study by Fang et al [7] showed that interdomain traffic distributions are highly non-uniform, an observation that has since been confirmed by others [1,11]. Feldmann et al [8] described a method to estimate web traffic demands using data from server logs at a large content delivery network.…”

“…Chang et al [4] propose a method to estimate interdomain traffic demand by estimating the importance of an AS in various roles -residential access, business access and web hosting. In contrast, our work aims to extract relevant statistical properties of the ITM from direct measurements, which resembles the approach in [11] for intradomain traffic. Sen et al [12] analysed P2P traffic in large networks.…”

Optimal monitoring in large networks by Successive c-optimal Designs

“…Since this network is only partially instrumented with Netflow (we dispose of Netflow measurements on 34 out of 116 routers), we simulated the missing data for the sake of experiments, by following the instructions of [20]. Namely, we noticed that the fit of the partially available data with a lognormal distribution was very good, so we simulated the missing flows with respect to this distribution, and we assigned them to the non-measured OD pairs of the network thanks to a heuristic procedure based on the topology of the network [20]. The data has a resolution of 2 hours and was collected during 40 hours, and so we track the flow volumes over 20 time steps.…”

Balancing performance, robustness and flexibility in routing systems

“…To capture random fluctuations in the intensity of traffic between individual SD pairs, we rely on a Gaussian model that has been shown appropriate in modeling such estimation errors [3,15]. This gives actual traffic intensities of the form r D (s, t) = r D (s, t) + N (0, εr D [s, t]) (11) r T (s, t) = r T (s, t) + N (0, εr T [s, t]) (12) where N (0, σ) denotes a normally distributed random variable with zero mean and standard deviation σ. ε controls the magnitude of possible traffic fluctuations.…”