A case for conservative workload modeling: Parallel job scheduling with daily cycles of activity

Abstract

Computer workloads have many attributes. When modeling these workloads it is often difficult to decide which attributes are important, and which can be abstracted away. In many cases, the modeler only includes attributes that are believed to be important, and ignores the rest. We argue, however, that this can lead to impaired workloads and unreliable system evaluations. Using parallel job scheduling as a case study, and daily cycles of activity as the attribute in dispute, we present two schedulers whose simulated performance seems identical without cycles, but then becomes significantly different when daily cycles are included in the workload. We trace this to the ability of one scheduler to prioritize interactive jobs, which leads to implicitly delaying less critical work to nighttime, when it can utilize resources that otherwise would have been left idle. Notably, this was not a design feature of this scheduler, but rather an emergent property that was not anticipated in advance.

Workload Modeling for Computer Systems Performance Evaluation

“…Importantly, not every workload feature has the same effect: in some cases it is one specific workload feature that is Version 1.0.3, typeset on the most important. The problem is that it is not always obvious in advance which feature is the most important, and even if it seems obvious, we might be wrong [237,438,248]. This motivates the practice of conservative workload modeling, where an attempt is made to correctly model all known workload features, regardless of their perceived importance [248].…”

“…The problem is that it is not always obvious in advance which feature is the most important, and even if it seems obvious, we might be wrong [237,438,248]. This motivates the practice of conservative workload modeling, where an attempt is made to correctly model all known workload features, regardless of their perceived importance [248]. Alternatively, it motivates the use of real workloads to drive evaluations, because real workloads may contain features that we do not know about and therefore cannot model.…”

“…Rather, it is the desire to capture a certain workload feature in a way that is good enough for a specific evaluation. However, if the effect of different workload features is not known, it is safer to try and mimic the observed workload as closely as possible [248].…”

“…However, in non-interactive settings the mere existence of a non-peak period which can be used to make up for overload during the peak hours can make a big difference [248]. When interactive and batch workloads are mixed, delaying batch work and executing it at night may free up important resources for better support of the interactive work during the day.…”

“…Many workload models ignore this feature and focus on the daytime workload, assuming that it is stationary. However, when the workload includes items whose duration is on the scale of hours (such as parallel jobs), the daily cycle cannot be ignored [248]. Also, in some cases it is wrong to assume that the daytime workload is more important; for example, in web search workloads the activity of home users in the evening and at night is no less important than the activity of professionals during the day [63].…”