Near Optimal Coflow Scheduling in Networks

Abstract

The coflow scheduling problem has emerged as a popular abstraction in the last few years to study data communication problems within a data center [6]. In this basic framework, each coflow has a set of communication demands and the goal is to schedule many coflows in a manner that minimizes the total weighted completion time. A coflow is said to complete when all its communication needs are met. This problem has been extremely well studied for the case of complete bipartite graphs that model a data center with full bisection bandwidth and several approximation algorithms and effective heuristics have been proposed recently [1,2,29].In this work, we study a slightly different model of coflow scheduling in general graphs (to capture traffic between data centers [15,29]) and develop practical and efficient approximation algorithms for it. Our main result is a randomized 2 approximation algorithm for the single path and free path model, significantly improving prior work. In addition, we demonstrate via extensive experiments that the algorithm is practical, easy to implement and performs well in practice.

Network-Aware Locality Scheduling for Distributed Data Operators in Data Centers

“…As we have described, the data locality scheduling optimization for both the aggregation and join operator can be converted into the MILP problem as presented in the optimization model P 2 . It is obvious that our optimization is more complex than a single coflow scheduling, which actually can be mapped to an open-shop scheduling problem [11], [26], [27], of which the computational complexity is NP-hard 2 . From a theoretical perspective, our problem solving time is exponential time.…”

Scheduling Flows on a Switch to Optimize Response Times

“…Approximation algorithms for average completion time of co-flows on a non-blocking switch are given in [1,38,49,48]. Scheduling over general network topologies is studied in [15,32,51], including approximation algorithms for average completion time.…”

“…The variable x e,t denotes the fraction of flow e scheduled in round t. Constraint (14) ensures that the total size of all edges adjacent to a port that are scheduled in a round is no more than the port's capacity. Constraint (15) ensures that all edges are scheduled.…”

“…Let d max = max e∈F {d e }. Let A and b be identical to A LP and b LP except that all rows corresponding to Constraint (16) have been removed, and all values in rows corresponding to constraint (15) have been multiplied by d max and made negative. Let ∆ = d max .…”

“…A flow is a particular set of application traffic between two endpoints that receive the same forwarding decisions. As a consequence of the shift towards centralized flow-based control, efficient algorithms for scheduling and routing of flows and their variants have gained prominent importance [15,47,11,39,29].…”