Asymptotically Optimal Approximation Algorithms for Coflow Scheduling

Abstract

Many modern datacenter applications involve large-scale computations composed of multiple data flows that need to be completed over a shared set of distributed resources. Such a computation completes when all of its flows complete. A useful abstraction for modeling such scenarios is a coflow, which is a collection of flows (e.g., tasks, packets, data transmissions) that all share the same performance goal.In this paper, we present the first approximation algorithms for scheduling coflows over general network topologies with the objective of minimizing total weighted completion time. We consider two different models for coflows based on the nature of individual flows: circuits, and packets. We design constant-factor polynomial-time approximation algorithms for scheduling packet-based coflows with or without given flow paths, and circuit-based coflows with given flow paths. Furthermore, we give an O(log n/ log log n)approximation polynomial time algorithm for scheduling circuitbased coflows where flow paths are not given (here n is the number of network edges).We obtain our results by developing a general framework for coflow schedules, based on interval-indexed linear programs, which may extend to other coflow models and objective functions and may also yield improved approximation bounds for specific network scenarios. We also present an experimental evaluation of our approach for circuit-based coflows that show a performance improvement of at least %22 on average over competing heuristics.

Near Optimal Coflow Scheduling in Networks

“…In the single path model, each flow f i j specifies a path p i j from source s i j ∈ V to sink t i j ∈ V so that the flow can only be routed along that path. This is exactly the "circuit-based coflows with paths given" model studied by Jahanjou et al [15].…”

“…The main result of this paper is a unified, tight 2-approximation algorithm for the coflow scheduling problem in both the single path model and the free path model when all release times and demands are polynomially sized, and a (2+ϵ)-approximation when the release times and demands can be super-polynomial. This improves upon the 17.6 approximation given by Jahanjou et al [15] for the single path model, and is the first approximation algorithm for the free path model (introduced by You and Chowdhury [29]).…”

“…The idea of scheduling coflows was first introduced by Chowdhury and Stoica [6]. Since then, it has been a hot topic in both the systems [7,19,20,30] and the theory [2,15,18,24,28] communities. Most theoretical research has focused on coflow scheduling in the switch model, where the communication graph is a complete, bipartite graph.…”

“…In order to solve these problems, a slightly different model of coflow scheduling was proposed by Jahanjou et al [15], which assumes that the underlying connection between machines is an arbitrary graph rather than a complete bipartite graph. Each node can be a machine, a datacenter or an exchange point (switch, router, etc.…”

Efficient Scheduling of Weighted Coflows in Data Centers

“…Since now, some existing literatures [11], [15], [27], [42] have already considered average weight coflow completion optimizaiton. However, these methods always try to solve a convex optimization problem, which may cause long computation time and it is impossible to use in the real world.…”