On Simulation and Design of Parallel-Systems Schedulers: Are We Doing the Right Thing?

Abstract

Abstract-It is customary to use open-system, trace-driven simulations to evaluate the performance of parallel-system schedulers. As a consequence, all schedulers have evolved to optimize the packing of jobs in the schedule, as a mean to improve a number of performance metrics that are conjectured to be correlated with user satisfaction, with the premise that this will result in a higher productivity in reality. We argue that these simulations suffer from severe limitations that lead to suboptimal scheduler designs, and to even dismissing potentially good design alternatives. We propose an alternative simulation methodology called site-level simulation, in which the workload for the evaluation is generated dynamically by user-models that interact with the system. We present a novel scheduler called CREASY that exploits knowledge on user behavior to directly improve user satisfaction, and compare its performance to the original, packing-based EASY scheduler. We show that user productivity improves by up to 50% under the user-aware design, while according to the conventional metrics, performance may actually degrade.Index Terms-Parallel job scheduling, trace-driven simulations, open-system model, user behavior, feedback.

Workload Modeling for Computer Systems Performance Evaluation

“…Shmueli and Feitelson elaborate a model for daytime users. In their model, users have a propensity to leave the system around 6 PM and return between 8 and 10 AM the next day [615,248]. This is done by keeping track of wallclock time when generating new arrivals.…”

“…For example, Shmueli and Feitelson propose a model in which users have a propensity to leave the system around 6 PM and return around 8 AM the next day [615,248]. Zilber at al.…”

“…Using the above, Shmueli also proposed a model for the effect of feedback on sessions. Setting the threshold representing session breaks at 20 minutes, the probability of not breaking the session (that is, the probability that the ensuing think time is shorter than 20 minutes) was found to depend on the response time according to Pr(continue) = 0.8 0.05 r + 1 where r is the response time in minutes [615]. But this needs to be combined with data about session starts and other aspects of user behavior in order to create a realistic model [750].…”

“…Little if any work has actually been done so far on modeling all these aspects of user behavior. One issue that has been modeled, however, is user reaction to system performance [614,615]. One question asks exactly what metric of performance really affects user behavior -for example, does response time capture performance as it is perceived by human users, or should we use slowdown instead?…”

“…where r is the response time in minutes [615]. Thus the probability of continuing to submit jobs in the same session is 0.8 when response is instantaneous, and it drops to 0.2 if the response time is one hour.…”