Traffic Engineering in Packet/Circuit Hybrid Networks

Abstract

Scientific computing applications, used in fields such as high-energy physics, climate science, genomics, etc., generate large (tera-to peta-byte sized) data sets. To move these heavy-hitter datasets fast, supercomputing sites invest in high-end clusters that can sustain high-speed transfers. Such large-sized, high-speed transfers are called α flows. These flows can have adverse effects on packet delays of real-time flows as α flows being bursty in nature can cause router buffer buildups.To enable the support of both α flows and real-time flows on the same network infrastructure while meeting quality-of-service (QoS) requirements of both types of flows, we propose a Hybrid Network Traffic Engineering System (HNTES). HNTES performs two tasks, 1) automatic identification of α flows at a core provider network's ingress routers, with no requirement of modifying end-user applications, 2) redirects these flows to traffic-engineered QoS-controlled virtual circuits.This dissertation describes two versions of the HNTES design. The first design explored the possibility of implementing an online mechanism that identifies alpha flows from live traffic. But the design proved to be cost prohibitive. Our second design uses an offline approach by analyzing NetFlow reports of completed flows to determine the IP addresses of source-destination pairs that move large datasets at high rates. These extracted IP addresses are used to set firewall filters in ingress routers to capture future alpha flows for redirection. This solution is based on a hypothesis that alpha flows are repeatedly created by the same source-destination hosts. NetFlow data for a 7-month period was obtained from an ESnet router to test the hypothesis, which proved to be true. It showed that had HNTES been deployed at the start of this period, over 91% of data that appeared in bursts from b Abstract c α flows would have been redirected. The final contribution of this thesis resulted from an experimental study of different scheduling and policing mechanisms implemented in routers.The objective was to find a suitable combination of mechanisms that achieved dual goals: reducing delay and jitter of real-time delay-sensitive flows, while at the same time allowing alpha flows to achieve high throughput. The best combination was found to be a no-policing, two-queue solution with priority and weighted fair queueing forms of packet scheduling. This result influenced the configuration of routers of a US backbone network provider. AcknowledgementsI would like to express my deepest gratitude to my advisor, Prof. Malathi Veeraraghavan, for her consistent guidance, support and patience. Prof. Veeraraghavan has provided an excellent example for me in so many ways through her breadth and depth of knowledge, intellectual curiosity, critical thinking, and most importantly, through her research attitude.It has been an invaluable experience to learn so much from her.

Traffic Engineering and Characterization of High-Rate Large-Sized Flows

“…It communicates with the routers, IDC, and NetFlow collector within its own network (as illustrated in Fig. 2.1 [29,31]). Its functions are described below.…”

“…Its functions are described below. [29,31] suming that HNTES runs on a nightly basis, it creates a list of α prefix IDs to store in a set F i , where i is a per-day index.…”