Performance analysis of work-conserving schedulers for minimizing total flow-time with phase precedence

Abstract

Abstract-We consider the problem of minimizing the total flow time of multiple jobs in a pool of multiple homogenous machines, where the jobs arrive over time and have to be served with phase precedence. This is a common occurrence in job scheduling for the increasingly popular data center oriented systems, where jobs need to be processed through a MAP and Reduce procedure before leaving the system. For this problem, one can construct an arrival pattern such that no scheduler can achieve a constant competitive ratio. However, what we find is that by using a slightly weaker metric of performance, which we call the efficiency ratio. We say that a scheduler achieves an efficiency ratio of γ when the flow time incurred by that scheduler divided by the minimum flow time achieved over all possible schedulers is less than or equal to γ almost surely, when the time slots or job arrivals go to infinity. We show a surprising property that any work-conserving scheduler for the flow time problem with phase precedence has a constant efficiency ratio in both preemptive and non-preemptive scenarios under some weak assumptions. We provide numerical results to support our analysis.

A new analytical technique for designing provably efficient MapReduce schedulers

“…However, the practical cost corresponding to the delay (or storage) of jobs are directly related to the total flowtime, not the completion time. Although the optimal solution is the same for these two optimization problems, the efficiency ratio obtained from minimizing the total flow-time will be much looser than the efficiency ratio obtained from minimizing the total completion time (details are shown in technical report [8]). …”

“…We obtain similar results to Theorem 2 and 3 with different expressions. More details are given in technical report [8].…”

“…9, where the workload distribution of Map as U [10,50] and the workload distribution of Reduce as U [10,20]. More simulations of convergence are shown in technical report [8].…”

“…Proof: The proof of NP-completeness follows a fairly standard reduction from 3-Partition and is described in technical report [8].…”