Efficient and Scalable Execution of Fine-Grained Dynamic Linear Pipelines

Abstract

We present Pipelite, a dynamic scheduler that exploits the properties of dynamic linear pipelines to achieve high performance for fine-grained workloads. The flexibility of Pipelite allows the stages and their data dependences to be determined at runtime. Pipelite unifies communication, scheduling, and synchronization algorithms with suitable data structures. This unified design introduces the local suspension mechanism and a wait-free enqueue operation, which allow efficient dynamic scheduling. The evaluation on a 44-core machine, using programs from three widely used benchmark suites, shows that Pipelite implies low overhead and significantly outperforms the state of the art in terms of speedup, scalability, and memory usage.

Chunking for Dynamic Linear Pipelines

“…The scheduling entity of Pipelite [18] and Piper [13] is a task that represents a single loop iteration. Both systems execute tasks either to completion or until they encounter a dependent stage iteration that needs data that are not available.…”

“…Pipelite [18] and Piper [13] are dynamic systems that efficiently handle dynamic linear pipelines, and they achieve load-balancing. In contrast to earlier systems [6,12,17,26], both Pipelite and Piper perform dynamic mapping of stages onto threads to achieve load-balancing.…”

“…To evaluate the performance of chunking, we implemented the described algorithm in two systems, a static scheduler for dynamic linear pipelines and the dynamic scheduler of Pipelite [18]. The experimental results obtained on a 44-core machine, using programs selected from three popular collections, show that the algorithm for chunking significantly reduces the overhead of both the static and the dynamic scheduler for the execution of fine-grained dynamic linear pipelines with an average execution time of less than 100μsec per loop iteration.…”

“…These constraints are related to the scheduling algorithm and have been used by the state-of-theart [13,18], but they are orthogonal to the contribution of this article. In particular, the chunking algorithm determines the execution order of stage iterations for a chunk of loop iterations.…”

“…Parallelism is achieved by overlapping the execution of different chunks, and the execution order is determined by the scheduling algorithm. Therefore, we assume the scheduling constraints of the state-of-the-art, and a more detailed description of the flexible structure supported by dynamic linear pipelines can be found in References [13,18]. In addition, the components of the state-of-the-art that are related to the scheduling algorithm are unnecessary for the chunking algorithm.…”