Deterministic scale-free pipeline parallelism with hyperqueues

Vandierendonck, Hans; Chronaki, Kallia; Nikolopoulos, Dimitrios S.

doi:10.1145/2503210.2503233

Cited by 9 publications

(14 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Vandierendonck et al [45] present hyperqueues, a programming abstraction of queues that allows the expression of deterministic and scale-free pipeline parallelism. However, hyperqueues cannot express the flexible structure of dynamic linear pipelines supported by Pipelite.…”

Section: Related Workmentioning

confidence: 99%

Efficient and Scalable Execution of Fine-Grained Dynamic Linear Pipelines

Mastoras

Groß

2019

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

We present Pipelite, a dynamic scheduler that exploits the properties of dynamic linear pipelines to achieve high performance for fine-grained workloads. The flexibility of Pipelite allows the stages and their data dependences to be determined at runtime. Pipelite unifies communication, scheduling, and synchronization algorithms with suitable data structures. This unified design introduces the local suspension mechanism and a wait-free enqueue operation, which allow efficient dynamic scheduling. The evaluation on a 44-core machine, using programs from three widely used benchmark suites, shows that Pipelite implies low overhead and significantly outperforms the state of the art in terms of speedup, scalability, and memory usage.

show abstract

Section: Related Workmentioning

confidence: 99%

Efficient and Scalable Execution of Fine-Grained Dynamic Linear Pipelines

Mastoras

Groß

2019

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

show abstract

“…We have analyzed the performance of these benchmarks using our runtime system in other work [Vandierendonck et al 2013], where we have also proposed new programming language concepts to accelerate these applications and improve their programmability. In this work, we evaluate the baseline versions using input, output, and in/out annotations.…”

Section: Benchmarksmentioning

confidence: 99%

“…The parallelism exploited in ferret consists of a five-stage pipeline where images are processed and compared in successive steps. Ferret's performance scales nearly linearly with an increasing number of cores [Vandierendonck et al 2013]. Ferret has fine-grain tasks, which causes differences between the dependence tracking schemes to show up in the overall execution time.…”

Section: Benchmarksmentioning

confidence: 99%

See 1 more Smart Citation

Analysis of dependence tracking algorithms for task dataflow execution

Vandierendonck

Tzenakis

Nikolopoulos

2013

ACM Trans. Archit. Code Optim.

Self Cite

View full text Add to dashboard Cite

Processor architectures has taken a turn toward many-core processors, which integrate multiple processing cores on a single chip to increase overall performance, and there are no signs that this trend will stop in the near future. Many-core processors are harder to program than multicore and single-core processors due to the need for writing parallel or concurrent programs with high degrees of parallelism. Moreover, many-cores have to operate in a mode of strong scaling because of memory bandwidth constraints. In strong scaling, increasingly finer-grain parallelism must be extracted in order to keep all processing cores busy.Task dataflow programming models have a high potential to simplify parallel programming because they alleviate the programmer from identifying precisely all intertask dependences when writing programs. Instead, the task dataflow runtime system detects and enforces intertask dependences during execution based on the description of memory accessed by each task. The runtime constructs a task dataflow graph that captures all tasks and their dependences. Tasks are scheduled to execute in parallel, taking into account dependences specified in the task graph.Several papers report important overheads for task dataflow systems, which severely limits the scalability and usability of such systems. In this article, we study efficient schemes to manage task graphs and analyze their scalability. We assume a programming model that supports input, output, and in/out annotations on task arguments, as well as commutative in/out and reductions. We analyze the structure of task graphs and identify versions and generations as key concepts for efficient management of task graphs. Then, we present three schemes to manage task graphs building on graph representations, hypergraphs, and lists. We also consider a fourth edgeless scheme that synchronizes tasks using integers. Analysis using microbenchmarks shows that the graph representation is not always scalable and that the edgeless scheme introduces least overhead in nearly all situations.

show abstract

“…We study the impact of performance on two benchmarks with higher complexity, namely ferret and dedup, taken from the PARSEC benchmark suite [Bienia 2011]. We have analyzed the performance of these benchmarks using our runtime system in other work [Vandierendonck et al 2013], where we have also proposed new programming language concepts to accelerate these applications and improve their programmability. In this work, we evaluate the baseline versions using input, output, and in/out annotations.…”

Section: Benchmarksmentioning

confidence: 99%

Analysis of dependence tracking algorithms for task dataflow execution

Vandierendonck¹,

Tzenakis²,

Nikolopoulos³

2013

TACO

Self Cite

View full text Add to dashboard Cite

Processor architectures has taken a turn toward many-core processors, which integrate multiple processing cores on a single chip to increase overall performance, and there are no signs that this trend will stop in the near future. Many-core processors are harder to program than multicore and single-core processors due to the need for writing parallel or concurrent programs with high degrees of parallelism. Moreover, many-cores have to operate in a mode of strong scaling because of memory bandwidth constraints. In strong scaling, increasingly finer-grain parallelism must be extracted in order to keep all processing cores busy. Task dataflow programming models have a high potential to simplify parallel programming because they alleviate the programmer from identifying precisely all intertask dependences when writing programs. Instead, the task dataflow runtime system detects and enforces intertask dependences during execution based on the description of memory accessed by each task. The runtime constructs a task dataflow graph that captures all tasks and their dependences. Tasks are scheduled to execute in parallel, taking into account dependences specified in the task graph. Several papers report important overheads for task dataflow systems, which severely limits the scalability and usability of such systems. In this article, we study efficient schemes to manage task graphs and analyze their scalability. We assume a programming model that supports input, output, and in/out annotations on task arguments, as well as commutative in/out and reductions. We analyze the structure of task graphs and identify versions and generations as key concepts for efficient management of task graphs. Then, we present three schemes to manage task graphs building on graph representations, hypergraphs, and lists. We also consider a fourth edgeless scheme that synchronizes tasks using integers. Analysis using microbenchmarks shows that the graph representation is not always scalable and that the edgeless scheme introduces least overhead in nearly all situations.

show abstract

Deterministic scale-free pipeline parallelism with hyperqueues

Cited by 9 publications

References 27 publications

Efficient and Scalable Execution of Fine-Grained Dynamic Linear Pipelines

Efficient and Scalable Execution of Fine-Grained Dynamic Linear Pipelines

Analysis of dependence tracking algorithms for task dataflow execution

Analysis of dependence tracking algorithms for task dataflow execution

Contact Info

Product

Resources

About