Symbiotic routing in future data centers

Abu-LibdehHussam,; CostaPaolo,; RowstronAntony,; O'SheaGreg,; DonnellyAustin,

doi:10.1145/1851275.1851191

Cited by 93 publications

(58 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When source routing is used in SDN, using network tomography to calculate the link metrics could be a more efficient and light-weight method, compared with each router to directly measure and report link metrics. Third, many studies [15]- [18] show that source routing may possess lower latency or greater available bandwidth, by partially or fully specifying the routing paths taken by packets.…”

Section: B Problem Formulationmentioning

confidence: 99%

Scalpel: Scalable Preferential Link Tomography Based on Graph Trimming

Gao¹,

Dong²,

Wu³

et al. 2016

IEEE/ACM Trans. Networking

View full text Add to dashboard Cite

Inferring per-link metrics through aggregated path measurements, known as network tomography, is an effective way to facilitate various network operations, such as network monitoring, load balancing, and fault diagnosis. We study the problem of identifying additive link metrics of a set of interesting links from end-to-end cycle-free path measurements among selected monitors, i.e., preferential link tomography. Since assigning a node as a monitor usually requires non-negligible operational cost, we focus on assigning the minimum number of monitors (i.e., optimal monitor assignment) to identify all interesting links. By modeling the network as a connected graph, we propose Scalpel, a scalable preferential link tomography approach. Scalpel trims the original graph by a two-stage graph trimming algorithm and reuses an existing method to assign monitors in the trimmed graph. We theoretically prove Scalpel has several key properties: 1) the graph trimming algorithm in Scalpel is minimal in the sense that further trimming the graph does not reduce the number of monitors; 2) the obtained assignment is able to identify all interesting links in the original graph; and 3) an optimal monitor assignment in the graph after trimming is also an optimal monitor assignment in the original graph. We implement Scalpel and evaluate it based on both synthetic topologies and real network topologies. Compared with state-of-the-art, Scalpel reduces the number of monitors by 39.0% to 98.6% when 50% to 1% of all links are interesting links.Index Terms-Graph trimming, network measurement, preferential link tomography.

show abstract

Section: B Problem Formulationmentioning

confidence: 99%

Scalpel: Scalable Preferential Link Tomography Based on Graph Trimming

Gao¹,

Dong²,

Wu³

et al. 2016

IEEE/ACM Trans. Networking

View full text Add to dashboard Cite

show abstract

“…BCube [6], DCell [5], CamCube [19], BCN [9], SWDC [20], and Scafida [21] fall into such a category. In such settings, a commodity server with ServerSwitch [22] card acts as not only an end host but also a mini-switch.…”

Section: Data Center Networkingmentioning

confidence: 99%

Exploiting Efficient and Scalable Shuffle Transfers in Future Data Center Networks

Guo

Xie

Zhou

et al. 2015

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Distributed computing systems like MapReduce in data centers transfer massive amount of data across successive processing stages. Such shuffle transfers contribute most of the network traffic and make the network bandwidth become a bottleneck. In many commonly used workloads, data flows in such a transfer are highly correlated and aggregated at the receiver side. To lower down the network traffic and efficiently use the available network bandwidth, we propose to push the aggregation computation into the network and parallelize the shuffle and reduce phases. In this paper, we first examine the gain and feasibility of the in-network aggregation with BCube, a novel server-centric networking structure for future data centers. To exploit such a gain, we model the in-network aggregation problem that is NP-hard in BCube. We propose two approximate methods for building the efficient IRS-based incast aggregation tree and SRS-based shuffle aggregation subgraph, solely based on the labels of their members and the data center topology. We further design scalable forwarding schemes based on Bloom filters to implement in-network aggregation over massive concurrent shuffle transfers. Based on a prototype and large-scale simulations, we demonstrate that our approaches can significantly decrease the amount of network traffic and save the data center resources. Our approaches for BCube can be adapted to other servercentric network structures for future data centers after minimal modifications.

show abstract

“…Besides being widely used in the High Performance Computing (HPC), the torus-based interconnects have also recently been introduced to data center clusters, and the typical examples include CamCube [3], Small-World [10].…”

Section: Related Workmentioning

confidence: 99%

“…CamCube [3] is a shipping container sized DCN based on a 3D torus topology, and it is the first work that introduces the torus fabric into data center interconnects. CamCube is one prototype that utilizes a low-level link oriented API to allow applications to implement their own particular routing protocols to optimize the application-level performance.…”

Section: Related Workmentioning

confidence: 99%

“…The traditional hierarchical switched DCN topology, besides being very costly, suffers high oversubscription ratio towards higher layers leading to serious communication bottleneck. Thus, researchers proposed several more cost-effective DCN architectures such as BCube [8], DCell [7], SprintNet [13], CamCube [3], Small-World [10] and NovaCube [12], where these server-centric architectures abandon expensive high-end network switches by using only low-end switches or even no switches at all. In addition, the forwarding functionality is shifted to the servers, which helps achieve higher bisection bandwidth and better fault tolerance with richer connections.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

CLOT: A cost-effective low-latency overlaid torus-based network architecture for data centers

Wang

Xia

et al. 2015

2015 IEEE International Conference on Communications (ICC)

View full text Add to dashboard Cite

In this paper, we present the design, analysis, and implementation of a novel data center network architecture named CLOT, which delivers significant reduction in the network diameter, network latency, and infrastructure cost. CLOT is built based on a switchless torus topology by adding only a number of most beneficial low-end switches in a proper way. Forming the servers in close proximity of each other in torus topology well implements the network locality. The extra layer of switches largely shortens the average routing path length of torus network, which increases the communication efficiency. We show that CLOT can achieve lower latency, smaller routing path length, higher bisection bandwidth and throughput, and better fault tolerance compared to both conventional hierarchical data center networks as well as the recently proposed CamCube network. Coupled with the coordinate based translated IP addresses, the carefully designed POW routing algorithm helps CLOT achieve its maximum theoretical performance. The sufficient mathematical analysis and theoretical derivation prove both guaranteed and ideal performance of CLOT.

show abstract

Symbiotic routing in future data centers

Cited by 93 publications

References 20 publications

Scalpel: Scalable Preferential Link Tomography Based on Graph Trimming

Scalpel: Scalable Preferential Link Tomography Based on Graph Trimming

Exploiting Efficient and Scalable Shuffle Transfers in Future Data Center Networks

CLOT: A cost-effective low-latency overlaid torus-based network architecture for data centers

Contact Info

Product

Resources

About