Scalable Loop Self-Scheduling Schemes Implemented on Large-Scale Clusters

Abstract

Loops are the largest source of parallelism in many scientific applications. Parallelization of irregular loop applications is a challenging problem to achieve scalable performance on large-scale multi-core clusters. Previous research proposed an effective Master-Worker model on clusters for distributed selfscheduling schemes that apply to parallel loops with independent iterations. However, this model has not been applied to large-scale clusters. In this paper, we present an extension of the distributed self-scheduling schemes implemented in a hierarchical Master-Worker model. Our experiments with different self-scheduling schemes demonstrate good scalability when scaling upto 8,192 processors.

Scalable Loop Self-Scheduling Schemes for Large-Scale Clusters and Cloud Systems

“…We use the following two applications in this implementation [7]. The outer loops in these applications are partitioned using scheduling and the tasks are assigned to workers.…”

“…The hierarchical scheme is based on a supermaster/master/worker model which can reduce the communication overhead and synchronization overhead. Preliminary results have been published in [6,7]. We implemented these schemes on a large-scale cluster of Texas Advanced Computing Center, University of Texas at Austin.…”

Load-prediction scheduling algorithm for computer simulation of electrocardiogram in hybrid environments

“…Workloads are one of the important sources of parallelism in scientific computing programs and therefore a lot of research was focused in this area (Wu et al, 2012). A step's workload is called a parallelizable workload if there is no data dependency among all steps, i.e., workloads can be processed in any order or even simultaneously (Han and Chronopoulos, 2013). The order of workload can be roughly divided into four kinds as shown in Fig.…”

“…Self-scheduling is a dynamic scheduling technique, through which idling processors can autonomously access a global data structure to obtain additional tasks (Hu et al, 2010). Various selfscheduling schemes, such as pure self-scheduling (PSS), factoring self-scheduling (FSS), guided self-scheduling (GSS), and trapezoid self-scheduling (TSS), have been proven successful for shared memory multiprocessor systems (Han and Chronopoulos, 2013). All selfscheduling schemes except PSS reduce dynamic scheduling overhead by reducing atomic operation times.…”