Rethinking the pipeline as object-oriented states with transformations

MacDonald, Steve; Szafron, Duane; Schaeffer, Jonathan

doi:10.1109/hips.2004.1299186

Cited by 21 publications

(18 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pipeline stages may be seen as transformations on work items [33]. Threads pick work items from queues holding work items from various stages in the pipeline.…”

Section: Schedulingmentioning

confidence: 99%

Deterministic scale-free pipeline parallelism with hyperqueues

Vandierendonck

Chronaki

Nikolopoulos

2013

Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

Ubiquitous parallel computing aims to make parallel programming accessible to a wide variety of programming areas using deterministic and scale-free programming models built on a task abstraction. However, it remains hard to reconcile these attributes with pipeline parallelism, where the number of pipeline stages is typically hard-coded in the program and defines the degree of parallelism.This paper introduces hyperqueues, a programming abstraction that enables the construction of deterministic and scale-free pipeline parallel programs. Hyperqueues extend the concept of Cilk++ hyperobjects to provide thread-local views on a shared data structure. While hyperobjects are organized around private local views, hyperqueues require shared concurrent views on the underlying data structure. We define the semantics of hyperqueues and describe their implementation in a work-stealing scheduler. We demonstrate scalable performance on pipeline-parallel PARSEC benchmarks and find that hyperqueues provide comparable or up to 30% better performance than POSIX threads and Intel's Threading Building Blocks. The latter are highly tuned to the number of available processing cores, while programs using hyperqueues are scale-free.

show abstract

“…Pipeline stages may be seen as transformations on work items [33]. Threads pick work items from queues holding work items from various stages in the pipeline.…”

Section: Schedulingmentioning

confidence: 99%

Deterministic scale-free pipeline parallelism with hyperqueues

Vandierendonck

Chronaki

Nikolopoulos

2013

Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

show abstract

“…MacDonald et al [13] proposed the master-slave scheduling technique that TBB uses as its execution model for pipelines. They implemented their technique in the patternbased parallel programming system CO 2 P 3 S. The CO 2 P 3 S system enabled a top-down approach to developing parallel programs: a developer selected the parallel programming patterns (pipeline parallelism being one of the selections), configured the parameters and the system generates the skeleton code (Java) for the different components.…”

Section: Related Workmentioning

confidence: 99%

“…TBB's pipeline execution model is based on MacDonald's work [13]. This tasking model alleviates the rampup/ramp-down problem as the pipelines starts/ends while also providing better load-balancing by allocating more tasks to different filters dynamically.…”

Section: Filter1mentioning

confidence: 99%

Expressing pipeline parallelism using TBB constructs

Reed

Chen

Johnson

2011

Proceedings of the Compilation of the Co-Located Workshops on DSM'11, TMC'11, AGERE! 2011, AOOPES'11, NEAT'11, &Amp; VMIL'11

View full text Add to dashboard Cite

Task-based libraries such as Intel's Threading Building Blocks (TBB) provide higher levels of abstraction than threads for parallel programming. Work remains, however, to determine how straightforward it is to use these libraries to express various patterns of parallelism. This case study focuses on a particular pattern: pipeline parallelism. We attempted to transform three representative applicationscontent-based image retrieval, compression and video encoding -to pipelines using TBB. We successfully converted two of the three applications. In the successful cases we discuss our transformation process and contrast the expressivity and performance of our implementations to existing Pthreads versions; in the unsuccessful case, we detail what the challenges were and propose potential solutions.

show abstract

“…Iterative, multi-phase applications are frequently structured as nested models with an outer code controlling multiple phases each based on a possibly different parallel pattern. The graphical animation described in [6] implements expensive pipeline stages with masterworker model. The FLASH [3] code we will study later nests parallel recursive tree computations in an adaptive mesh refinement model.…”

Section: Computational Model Compositionmentioning

confidence: 99%