A simulation-based study of scheduling mechanisms for a dynamic cluster environment

Abstract

Scheduling of processes onto processors of a parallel machine has always been an important and challenging area of research. The issue becomes even more crucial and di cult as we gradually progress to the use of o -the-shelf workstations, operating systems, and high bandwidth networks to build cost-e ective clusters for demanding applications. Clusters are gaining acceptance not just in scienti c applications that need supercomputing power, but also in domains such as databases, web service and multimedia, which place diverse Quality-of-Service QoS demands on the underlying system. Further, these applications have diverse characteristics in terms of their computation, communication and I O requirements, making conventional parallel scheduling solutions, such as space sharing or coscheduling, an unattractive option. At the same time, leaving it to the native operating system of each node to make decisions independently can lead to ine ective use of system resources whenever there is communication. Instead, an emerging class of dynamic coscheduling mechanisms, that attempt to take remedial actions to guide the system towards coscheduled execution without requiring explicit synchronization, o er a lot of promise for cluster scheduling. Using a detailed simulator, this paper evaluates the pros and cons of di erent dynamic coscheduling alternatives, while comparing their advantages over traditional coscheduling and not performing any coordinated scheduling at all. The impact of dynamic job arrivals, job characteristics and di erent system parameters on these alternatives are evaluated in terms of several performance criteria.

Evaluation of fault-tolerant policies using simulation

“…Our goal is to study and evaluate different FT policies and not just periodic checkpoint. Zhang et al [10] [11] study performance metrics such as resource utilization, job slowdown, for scheduling strategies and not for fault-tolerant mechanisms.…”

ClusterSchedSim: A Unifying Simulation Framework for Cluster Scheduling Strategies

“…This problem is accentuated when we multiprogram a large number of tasks which can involve considerable swapping overheads. For this reason, several systems (e.g., [29,32] ) limit the multiprogramming level (MPL) so that swapping overheads are kept to a minimum. Under the same rationale, we use the multiprogramming level directly to modulate the memory usage rather than explicitly model the memory consumption.…”

“…Further, the tasks from the same job must be scheduled into the same row, which makes it more challenging. We have implemented the exact co-scheduling heuristics that are proposed in our earlier work [30,29,28,33,32].…”

“…For instance, our experiments show that an MPL level between 5 and 16 is optimal for some dynamic co-scheduling schemes [32].…”

Um estudo comparativo de cargas de trabalho e políticas de escalonamento para aplicações paralelas em clusters e grids computacionais

“…Entretanto, a grande maioria dos trabalhos utiliza apenas os termos NOW (YANG et al, 2003;KARATZA;HILZER, 2003;SHIM, 2004) e COW (CHIOLA; CIACCIO, 2000;ZHANG et al, 2000;ZHANG et al, 2001;KARATZA, 2002a).…”