Efficient Work Stealing for Fine Grained Parallelism

SUMMARYLazy-task creation is an efficient method of overcoming the overhead of the grain-size problem in parallel computing. Work stealing is an effective load balancing strategy for parallel computing. In this paper, we present dynamic work stealing strategies in a lazy-task creation technique for efficient fine-grain task scheduling. The basic idea is to control load balancing granularity depending on the number of task parents in a stack. The dynamic-length strategy of work stealing uses run-time information, which is information on the load of the victim, to determine the number of tasks that a thief is allowed to steal. We compare it with the bottommost first work stealing strategy used in StackThread/MP, and the fixed-length strategy of work stealing, where a thief requests to steal a fixed number of tasks, as well as other multithreaded frameworks such as Cilk and OpenMP task implementations. The experiments show that the dynamic-length strategy of work stealing performs well in irregular workloads such as in UTS benchmarks, as well as in regular workloads such as Fibonacci, Strassen's matrix multiplication, FFT, and Sparse-LU factorization. The dynamic-length strategy works better than the fixed-length strategy because it is more flexible than the latter; this strategy can avoid load imbalance due to overstealing.

show abstract

“…Faxen introduced the load-balancing concept in his paper [21]. However, we redefine load-balancing granularity in Eq.…”

Section: Controlling Load Balance Granularitymentioning

confidence: 99%

Dynamic Multiple Work Stealing Strategy for Flexible Load Balancing

Adnan

Sato

2012

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…This technique is called leapfrogging [16]. When stealing from random workers instead, the size of the task pool of each worker could grow beyond the size needed for complete sequential execution [8]. Using leapfrogging rather than stealing from random workers thus limits the space requirements of the task pools to those of sequential execution.…”

Section: Work-stealingmentioning

confidence: 99%

“…To virtually eliminate the overhead of task creation for tasks that are never stolen, Faxén proposed a direct task stack, storing tasks instead of pointers in the work queue, implemented in Wool [7,8]. Rather than synchronizing with thieves on the metadata of the queue (e.g.…”

Section: Work-stealing Dequesmentioning

confidence: 99%

See 2 more Smart Citations

Lace: Non-blocking Split Deque for Work-Stealing

Dijk

Pol

2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Work-stealing is an efficient method to implement load balancing in fine-grained task parallelism. Typically, concurrent deques are used for this purpose. A disadvantage of many concurrent deques is that they require expensive memory fences for local deque operations. In this paper, we propose a new non-blocking work-stealing deque based on the split task queue. Our design uses a dynamic split point between the shared and the private portions of the deque, and only requires memory fences when shrinking the shared portion. We present Lace, an implementation of work-stealing based on this deque, with an interface similar to the work-stealing library Wool, and an evaluation of Lace based on several common benchmarks. We also implement a recent approach using private deques in Lace. We show that the split deque and the private deque in Lace have similar low overhead and high scalability as Wool.

show abstract

A comparative performance study of common and popular task‐centric programming frameworks

Podobas

Brorsson

Faxén

2013

Concurrency and Computation

View full text Add to dashboard Cite

Programmers today face a bewildering array of parallel programming models and tools, making it difficult to choose an appropriate one for each application. An increasingly popular programming model supporting structured parallel programming patterns in a portable and composable manner is the task-centric programming model. In this study, we compare several popular task-centric programming frameworks, including Cilk Plus, Threading Building Blocks, and various implementations of OpenMP 3.0. We have analyzed their performance on the Barcelona OpenMP Tasking Suite benchmark suite both on a 48-core AMD Opteron 6172 server and a 64-core TILEPro64 embedded many-core processor. Our results show that the OpenMP offers the highest flexibility for programmers, and this flexibility comes to a cost. Frameworks supporting only a specific and more restrictive model, such as Cilk Plus and Threading Building Blocks, are generally more efficient both in terms of performance and energy consumption. However, Intel's implementation of OpenMP tasks performs the best and closest to the specialized run-time systems. Mercurium is the source-to-source compiler used in conjunction with Nanos++. † † Because the (not-entirely unexpected) lack of a TILEPro64 back-end in the Intel compiler. ‡ ‡ This means that to synchronize with N tasks, the programmer need to explicitly use SYNC N times. PERFORMANCE IN TASK-CENTRIC PROGRAMMING FRAMEWORKS 13 Figure 5. Memory footprint for each run-time system implementation when normalized against a serial execution. The base is the serial executing compiled with GCC.example, Fibonacci and N-queens. Intel's TBB is the implementation, which has the largest memory footprint of all models.4.4.6. Embedded power measurements. Power consumption and energy has risen to become as important metrics as performance, in particular on embedded devices. We have measured the power and energy consumed by the application under different run-time systems on the TILEPro64. The reason we have not performed for the Opteron systems is because we found it to be much more difficult to isolate the effect of the processors and memory on that machine, although it was relatively straight-forward on the TILEPro64. We used a data acquisition device (NI USB-6210) to perform the power measurements on the TILEPro64. Sampling the power consumption did in no way interfere with the program execution, as it was performed on a separate computer connected to the data acquisition device. This set-up is similar to the ones used by Själander et al. [44]. The measured sampling frequency was 20 kHZ. We used a metric that we call speed-up power cost, which calculates the speed-up and application experiences for each added watt. EXPERIMENTAL RESULTS AND DISCUSSIONThis section presents the experimental results obtained from micro-benchmarks and other benchmarks according to the methodology from the previous section. Micro-benchmarksAll the micro-benchmark measurements were performed on the Opteron 6172 48-core system. A. PODOBAS, M. BRORSSON AND K.-F. FA...

show abstract

Efficient Work Stealing for Fine Grained Parallelism

Cited by 34 publications

References 19 publications

Dynamic Multiple Work Stealing Strategy for Flexible Load Balancing

Dynamic Multiple Work Stealing Strategy for Flexible Load Balancing

Lace: Non-blocking Split Deque for Work-Stealing

A comparative performance study of common and popular task‐centric programming frameworks

Contact Info

Product

Resources

About