A performance study on operator-based stream processing systems

Dayarathna, Miyuru; Takeno, Souhei; Suzumura, Toyotaro

doi:10.1109/iiswc.2011.6114204

Cited by 6 publications

(3 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main areas identified from the set of 50 resulting papers were: sensor networks, advertising, finance, telecommunication, social networks, synthetic applications, and network monitoring. The percentage of papers in our set belonging to the different areas is depicted in Figure 2 and the complete list can be found in Table 2 Area Papers Finance [25], [26], [27], [28] Network Monitoring [29], [30], [31], [32] Synthetic [33], [34], [35], [36], [37], [38] Traffic Monitoring [39], [40], [41], [42], [43] Advertising [29], [30], [15], [44], [45], [46], [47] Sensor Network [48], [49], [50], [51], [52], [53], [11], [54] Social Network [55], [56], [57], [58], [59], [60], [61], [62], [63], [18] Telecom…”

Section: A Application Domainsmentioning

confidence: 99%

DSPBench: A Suite of Benchmark Applications for Distributed Data Stream Processing Systems

et al. 2020

View full text Add to dashboard Cite

Section: A Application Domainsmentioning

confidence: 99%

DSPBench: A Suite of Benchmark Applications for Distributed Data Stream Processing Systems

et al. 2020

View full text Add to dashboard Cite

“…Performance of stream processing systems is a widely studied area. There have been multiple previous work on performance evaluation of stream processing systems [19,[21][22][23][24][25]. Akram et al studied the efficiency of multilayer software stack used for stream processing in a data center environment [26].…”

Section: Related Workmentioning

confidence: 99%

Energy consumption analysis of data stream processing: a benchmarking approach

Dayarathna

Wen

et al. 2016

Softw Pract Exp

Self Cite

View full text Add to dashboard Cite

Summary Energy efficiency of data analysis systems has become a very important issue in recent times because of the increasing costs of data center operations. Although distributed streaming workloads have increasingly been present in modern data centers, energy‐efficient scheduling of such applications remains as a significant challenge. In this paper, we conduct an energy consumption analysis of data stream processing systems in order to identify their energy consumption patterns. We follow stream system benchmarking approach to solve this issue. Specifically, we implement Linear Road benchmark on six stream processing environments (S4, Storm, ActiveMQ, Esper, Kafka, and Spark Streaming) and characterize these systems' performance on a real‐world data center. We study the energy consumption characteristics of each system with varying number of roads as well as with different types of component layouts. We also use a microbenchmark to capture raw energy consumption characteristics. We observed that S4, Esper, and Spark Streaming environments had highest average energy consumption efficiencies compared with the other systems. Using a neural networkbased technique with the power/performance information gathered from our experiments, we developed a model for the power consumption behavior of a streaming environment. We observed that energy‐efficient execution of streaming application cannot be specifically attributed to the system CPU usage. We observed that communication between compute nodes with moderate tuple sizes and scheduling plans with balanced system overhead produces better power consumption behaviors in the context of data stream processing systems. Copyright © 2016 John Wiley & Sons, Ltd.

show abstract

“…S4 [6], which allow to scale across a large number of hosts, such performance goals become more easily achievable. However, the required resources to process such tasks depend on many different factors like the streaming system and the use case [2]. In context of cloud computing a "pay per use" model is used.…”

Section: Introductionmentioning

confidence: 99%

Demo

Heinze

Meyer

Jerzak

et al. 2013

Proceedings of the 7th ACM International Conference on Distributed Event-Based Systems

View full text Add to dashboard Cite

In recent time due to the availability of cloud-based data streaming systems like Yahoo! S4 or Twitter Storm and virtually unlimited resources using a public cloud infrastructure it is possible to run stream processing tasks with a new dimension of computational complexity. However, the required resources in terms of CPU, memory, and network bandwidth differ depending on the use case and applied data streaming system. For the user of such a system this is directly visible in the monetary cost he has to spent for the used resources. Therefore, he would like to maximize the ratio between gained performance and his monetary cost.In our demonstration we present an approach to measure and estimate the monetary cost for data streaming systems. We present a general scheme to model monetary cost for any combination of a cloud-based data streaming system and a major public cloud provider. Our model can be used as a starting point for optimizing the ratio between monetary cost and performance of streaming systems in general.

show abstract

A performance study on operator-based stream processing systems

Cited by 6 publications

References 3 publications

DSPBench: A Suite of Benchmark Applications for Distributed Data Stream Processing Systems

DSPBench: A Suite of Benchmark Applications for Distributed Data Stream Processing Systems

Energy consumption analysis of data stream processing: a benchmarking approach

Demo

Contact Info

Product

Resources

About