Orchestrating Bulk Data Transfers across Geo-Distributed Datacenters

Wu, Yu; Zhang, Zhizhong; Wu, Chuan; Guo, Chuanxiong; Li, Zongpeng; Lau, Francis C. M.

doi:10.1109/tcc.2015.2389842

Cited by 76 publications

(53 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Unlike previous presented works, our work manages this tradeoff through system interfaces, which allows users to specify their preferences, and provides an efficient mechanism to find this tradeoff. Though Wu et al [17] proposed scheduling approaches for bulk data transfers with different urgency levels. In their work, the priority assigned to a data transfer only concerned the job scheduling without taking into account the time-cost tradeoff.…”

Section: Simulation Resultsmentioning

confidence: 99%

CoShare: A Cost-Effective Data Sharing System for Data Center Networks

Zhuang

Filali

Rahman

et al. 2015

2015 IEEE Conference on Collaboration and Internet Computing (CIC)

View full text Add to dashboard Cite

Abstract-Numerous research groups and other organizations collect data from popular data sources such as online social networks. This leads to the problem of data islands, wherein all this data is isolated and lying idly, without any use to the community at large. Using existing centralized solutions such as Dropbox to replicate data to all interested parties is prohibitively costly, given the large size of datasets. A practical solution is to use a Peer-to-Peer (P2P) approach to replicate data in a self-organized manner. However, existing P2P approaches focus on minimizing downloading time without taking into account the bandwidth cost. In this paper, we present CoShare, a P2P inspired decentralized cost effective sharing system for data replication. CoShare allows users to specify their requirements on data sharing tasks and maps these requirements into resource requirements for data transfer. Through extensive simulations, we demonstrate that CoShare finds the desirable tradeoffs for a given cost and performance while varying user requirements and request arrival rates.

show abstract

Section: Simulation Resultsmentioning

confidence: 99%

CoShare: A Cost-Effective Data Sharing System for Data Center Networks

Zhuang

Filali

Rahman

et al. 2015

2015 IEEE Conference on Collaboration and Internet Computing (CIC)

View full text Add to dashboard Cite

show abstract

“…However, the adoption of the SDN approach has quickly proven its benefits also in different scenarios with more relaxed requirements in terms of closeness and geographical centralization. For instance, SDN is exploited in wide area networks to efficiently interconnect different datacenters [10,11], eventually based on a multi-controller SDN architecture [12]. In addition, the SDN approach has been proposed also in scenarios differing from traditional datacenters, such as MANETs [13,14], vehicular networks [15], naval systems [16], and access/transport networks [6,10,17].…”

Section: A Novel Taxonomy For State-of-the-art Sdn Controllersmentioning

confidence: 99%

“…For instance, software-driven WAN (SWAN) [22] improves inter-datacenter network exploitation by centrally tuning when and how much traffic services generate and reconfiguring the data plane in relation to actual traffic demand. Wu et al [11] address the issue of managing bulk data transfers among geo-distributed datacenters hosted by a single cloud provider. To this purpose, they propose that a central SDN controller gathers information from distributed gateways to optimally schedule interdatacenter transfers based on per-chunk routing choices.…”

Section: A Novel Taxonomy For State-of-the-art Sdn Controllersmentioning

confidence: 99%

Multi-domain SDN controller federation in hybrid FiWi-MANET networks

Bellavista

Giannelli

Λάγκας³

et al. 2018

J Wireless Com Network

View full text Add to dashboard Cite

Traditionally, mobile ad hoc networks (MANET) and hybrid optical-wireless networks, also known as fiber-wireless (FiWi), have been considered disjointly and independently, with no synergic management solutions. The former has primarily focused on dispatching packets among mobile nodes in infrastructure-less and very dynamic environments, the latter on offering high-bandwidth and low-latency access to cellular-equipped mobile nodes. The recent advancements and penetration of software-defined networking (SDN) techniques have stimulated the adoption of SDN-based flexible monitoring and control also for the MANET and FiWi domains. In this perspective, the paper originally proposes a model and an architecture that loosely federate MANET and FiWi domains based on the interaction of MANET and FiWi controllers: the primary idea is that our MANET and FiWi federated controllers can seldom exchange monitoring data and control hints the one with the other, thus mutually enhancing their capability of packet management over hybrid FiWi-MANET networks. Our model is applied to several relevant use cases to practically point out the benefits of the proposal in terms of both load balancing and fairness improvements.

show abstract

“…The authors in [5] presented a processing system for executing a sequence of MapReduce jobs on Geo-distributed datacentres where the treating of jobs is optimized according to time and pecuniary cost. The authors in [6] proposed a dynamic bulk data transfer framework in geo-distributed data canters and planned its design and algorithms depending on Software Defined Network (SDN) architecture. In [7], the authors demonstrated the minimization of overall cost for big data placement, processing, and movement across geo-distributed datacentres.…”

Section: Introductionmentioning

confidence: 99%

Greening big data networks: Volume impact

Al-Salim

Ali

Lawey

et al. 2016

2016 18th International Conference on Transparent Optical Networks (ICTON)

View full text Add to dashboard Cite

Tremendous volumes generated by big data applications are starting to overwhelm data centers and networks. Traditional research efforts have determined how to process these vast volumes of data inside datacenters. Nevertheless, slight attention has addressed the increase in power consumption resulting from transferring these gigantic volumes of data from the source to destination (datacenters). An efficient approach to address this challenge is to progressively processing large volumes of data as close to the source as possible and transport the reduced volume of extracted knowledge to the destination. In this article, we examine the impact of processing different big data volumes on network power consumption in a progressive manner from source to datacenters. Accordingly, a noteworthy decrease for data transferred is achieved which results in a generous reduction in network power consumption. We consider different volumes of big data chunks. We introduce a Mixed Integer Linear Programming model (MILP) to optimize the processing locations of these volumes of data and the locations of two datacenters. The results show that serving different big data volumes of uniform distribution yields higher power saving compared to the volumes of chunks with fixed size. Therefore, we obtain an average network power saving of 57%, 48%, and 35% when considering the volumes of 10-220 (uniform) Gb, 110 Gb, and 50 Gb per chunk, respectively, compared to the conventional approach where all these chunks are processed inside datacenters only.

show abstract

Orchestrating Bulk Data Transfers across Geo-Distributed Datacenters

Cited by 76 publications

References 19 publications

CoShare: A Cost-Effective Data Sharing System for Data Center Networks

CoShare: A Cost-Effective Data Sharing System for Data Center Networks

Multi-domain SDN controller federation in hybrid FiWi-MANET networks

Greening big data networks: Volume impact

Contact Info

Product

Resources

About