Unfairness Metrics for Space-Sharing Parallel Job Schedulers

Abstract

Sociology, computer networking and operations research provide evidence of the importance of fairness in queuing disciplines. Currently, there is no accepted model for characterizing fairness in parallel job scheduling. We introduce two fairness metrics intended for parallel job schedulers, both of which are based on models from sociology, networking, and operations research. The first metric is motivated by social justice and attempts to measure deviation from arrival order, which is perceived as fair by the end user. The second metric is based on resource equality and compares the resources consumed by a job with the resources deserved by the job. Both of these metrics are orthogonal to traditional metrics, such as turnaround time and utilization.The proposed fairness metrics are used to measure the unfairness for some typical scheduling policies via simulation studies. We analyze the fairness of these scheduling policies using both metrics, identifying similarities and differences.

Web-Scale Job Scheduling

“…Maximizing utilization, in job scheduling, however, can be at odds with responsiveness and fairness [9,11,25]. In particular, Web schedulers have the choice of collocating jobs from different SLA tiers on the same compute nodes, or of allocating dedicated nodes for each desired tier.…”

On Simulation and Design of Parallel-Systems Schedulers: Are We Doing the Right Thing?

“…Furthermore, our scheduler actually inherits its backfilling algorithm from the EASY scheduler. In fact, the only difference between the two schedulers is in the way they prioritize the waiting jobs: while EASY accounts only for the jobs' arrival order in the interest of fairness [12], our scheduler tries to assess the criticality of the jobs for the users, and assigns its priorities accordingly. We therefore named our scheduler CREASY, with "CR" standing for CRiticality, and "EASY" to denote the backfilling algorithm internally used.…”

Job Scheduling with Lookahead Group Matchmaking for Time/Space Sharing on Multi-core Parallel Machines

“…The work in [11] measures overall fairness (in retro) of a job scheduler by considering the actual start time of a job vs. its virtual start time without effects from later arriving jobs. The slack approach in [17] tries to maintain relative fairness among jobs by dynamically calculating possible delays in the presence of different job priorities.…”