Generating synthetic task graphs for simulating stream computing systems

Abstract

Publication informationJournal of Parallel and Distributed Computing, 73 (10): 1362-1374 Publisher ElsevierItem record/more information http://hdl.handle.net/10197/9894 Publisher's statement þÿ T h i s i s t h e a u t h o r s v e r s i o n o f a w o r k t h a tThe UCD community has made this article openly available. Please share how this access benefits you. Your story matters! (@ucd_oa) Some rights reserved. For more information, please see the item record link above.Stream-computing is an emerging computational model for performing complex operations on and across multi-source, high-volume data flows. The pool of mature publicly available applications employing this model is fairly small, and therefore the availability of workloads for various types of applications is scarce.Thus, there is a need for synthetic generation of large-scale workloads to drive simulations and estimate the performance of stream-computing applications at scale. We identify the key properties shared by most task graphs of streamcomputing applications and use them to extend known random graph generation concepts with stream computing specific features, providing researchers with realistic input stream graphs. Our graph generation techniques serve the purpose of covering a disparity of potential applications and user input -Our first "domain-specific" framework exhibits high user-controlled configurability while the second "application-agnostic" framework only takes the number of vertices as an input.

Throughput-Driven Partitioning of Stream Programs on Heterogeneous Distributed Systems

“…Therefore, each application when deployed on different targets will have a different number of nodes and streams. To diversify the experiment, we use the stream graph generator described in [3] to create two synthetic stream graphs with approximately 500 and 1,000 vertices. They are called S500 and S1;000, respectively.…”

Cloud‐based data streams optimization

“…Algorithms in Ajwani et al (2013) and Guirado et al (2013), andSafaei et al (2012)) processed data streams based on task graph structure. In Safaei et al (2012), the dynamic tuple routing (DTR) algorithm was introduced for continuous queries parallel processing in a multiprocessing environment.…”

“…In Guirado et al (2013), they proposed the data parallel replication mechanism and the task copy replication mechanism (TCRM) to improve the performance of parallel and distributed data streams processing by obtaining the best task graph structure for data streams and applying concurrency processing on more than one instance of data. A graph-based framework was introduced in Ajwani et al (2013) for processing data streams over task graphs. Graph generation techniques were proposed to generate undirected task graphs correctly.…”

Co-optimizing application partitioning and network topology for a reconfigurable interconnect

“…Since we need a large number of application graphs of varying sizes to evaluate different settings in our experiments, our requirements can't all be met by the limited number of public datasets. Therefore, we focus on synthetic stream-computing task graphs and in particular, we use a graph generator specifically written for simulating streaming applications [19,20].…”