Adding data parallelism to streaming pipelines for throughput optimization

Abstract

The streaming model is a popular model for writing high-throughput parallel applications. A streaming application is represented by a graph of computation stages that communicate with each other via FIFO channels. In this paper, we consider the problem of mapping streaming pipelines -streaming applications where the graph is a linear chain -onto a set of computing resources in order to maximize its throughput. In a parallel setting, subsets of stages, called components, can be mapped onto different computing resources. The throughput of an application is determined by the throughput of the slowest component.Therefore, if some stage is much slower than others, then it may be useful to replicate the stage's code and divide its workload among two or more replicas in order to increase throughput. However, pipelines may consist of some replicable and some non-replicable stages. In this paper, we address the problem of mapping these partially replicable streaming pipelines onto both homogeneous and heterogeneous platforms so as to maximize throughput.We consider two types of platforms, homogeneous platforms -where all resources are identical, and hetero geneous platforms -where resources may have different speeds. In both cases, we consider two network topologies -unidirectional chain and clique. We provide polynomial time algorithms for mapping partially replicable pipelines onto unidirectional chains for both homogeneous and heterogeneous platforms. For homogeneous platforms, the algorithm for unidirectional chains generalizes to clique topologies. However, for heterogeneous platforms, map ping these pipelines onto clique topologies is NP-complete.We provide heuristics to generate solutions for cliques by applying our chain algorithms to a series of chains sampled from the clique. Our empirical results show that these heuristics rapidly converge to near-optimal solutions.

RaftLib

“…Streams that can be processed out of order are ideal candidates for the run-time to automatically parallelize. Li et al [35] describe algorithms for replicating kernels in a pipelined environment, both for homogeneous compute resources and for heterogeneous compute resources.…”

“…The placement of each kernel changes not only the throughput but also the latency of the overall application. In addition, it is often possible to replicate kernels (executing them in parallel) without altering the application semantics [35]. RaftLib exploits this ability to blend pipeline and data parallelism as well.…”

RaftLib: A C++ template library for high performance stream parallel processing

“…Understanding this information is critical to understanding the secondary effects that each decision has for the performance of an application. Within a streaming data-flow graph, it is often possible to replicate kernels (executing them in parallel) to enhance performance without altering the application semantics (Li et al, 2013). RaftLib exploits this ability to extract more pipeline and task parallelism at runtime (dynamically) without further input from the programmer.…”

Deadlock-free buffer configuration for stream computing

“…This "batching" idea has been implemented in some streaming computing systems [11,31]. Note amortizing communication overhead is just one way of improving throughput of streaming application, which can also be optimized in many other ways [12,19].…”