Workload resampling for performance evaluation of parallel job schedulers

Abstract

SUMMARYEvaluating the performance of a computer system is based on using representative workloads. Common practice is to either use real workload traces to drive simulations or use statistical workload models that are based on such traces. Such models allow various workload attributes to be manipulated, thus providing desirable flexibility, but may lose details of the workload's internal structure. To overcome this, we suggest to combine the benefits of real traces and flexible modeling. Focusing on the problem of evaluating the performance of parallel job schedulers, we partition the trace of submitted jobs into independent subtraces representing different users and then recombine them in various ways, while maintaining features such as long‐range dependence and the daily and weekly cycles of activity. This facilitates the creation of longer workload traces that enable longer simulations, the creation of multiple statistically similar workloads that can be used to gauge confidence intervals, the creation of workloads with different load levels, and increasing the frequency of specific events like large surges of activity. Copyright © 2014 John Wiley & Sons, Ltd.

Workload Modeling for Computer Systems Performance Evaluation

“…Similar results have been obtained for the workloads on file systems [303], parallel machines [669,749], and more. Although self-similarity is basically a statistical property, it is hard to reproduce by a simple sampling from a distribution.…”

“…For example, user resampling is a methodology to create multiple versions of the same basic workload by partitioning it into the subtraces representing different users, and then recombining them randomly to create new workloads. Interestingly, in the context of parallel job workloads at least, it has been shown that such manipulations do not affect the self-similarity of the workload [749]. This lends support to models such as the merged on-off processes described later.…”

“…Instead, the load is manipulated by a highlevel modification of the user population: adding more users increases the load, and removing users reduces the load [57,108,749]. Alternatively, load can be modified by changing the characteristics of the active users.…”

“…In a sense, user-based modeling is an extension of this practice, which makes it more explicit. Importantly, at least in parallel job workloads it has been shown that self-similarity is indeed captured by the combined activity of multiple users, even if they are mixed [749].…”

“…by mixing normal and abnormal users, user-based modeling enables such singular behavior to be modeled separately from the modeling of normal activity [749]. This leads to more representative models of normal behavior, as well as the option to control the characteristics of abnormal behavior that are desired for an evaluation.…”