A Performance Comparison of Open-Source Stream Processing Platforms

Lopez, Martin Andreoni; Lobato, Antonio Gonzalez Pastana; Duarte, Otto Carlos Muniz Bandeira

doi:10.1109/glocom.2016.7841533

Cited by 72 publications

(53 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Lopez et al [16] propose a benchmarking framework to assess the throughput performance of Apache Storm, Spark, and Flink under node failures. The key finding of their work is that Spark is more robust to node failures but it performs up to an order of magnitude worse than Storm and Flink.…”

Section: Related Workmentioning

confidence: 99%

Benchmarking Distributed Stream Data Processing Systems

Karimov

Rabl

Katsifodimos

et al. 2018

2018 IEEE 34th International Conference on Data Engineering (ICDE)

162

120

View full text Add to dashboard Cite

The need for scalable and efficient stream analysis has led to the development of many open-source streaming data processing systems (SDPSs) with highly diverging capabilities and performance characteristics. While first initiatives try to compare the systems for simple workloads, there is a clear gap of detailed analyses of the systems' performance characteristics. In this paper, we propose a framework for benchmarking distributed stream processing engines. We use our suite to evaluate the performance of three widely used SDPSs in detail, namely Apache Storm, Apache Spark, and Apache Flink. Our evaluation focuses in particular on measuring the throughput and latency of windowed operations, which are the basic type of operations in stream analytics. For this benchmark, we design workloads based on real-life, industrial use-cases inspired by the online gaming industry. The contribution of our work is threefold. First, we give a definition of latency and throughput for stateful operators. Second, we carefully separate the system under test and driver, in order to correctly represent the open world model of typical stream processing deployments and can, therefore, measure system performance under realistic conditions. Third, we build the first benchmarking framework to define and test the sustainable performance of streaming systems. Our detailed evaluation highlights the individual characteristics and use-cases of each system.

show abstract

Section: Related Workmentioning

confidence: 99%

Benchmarking Distributed Stream Data Processing Systems

Karimov

Rabl

Katsifodimos

et al. 2018

2018 IEEE 34th International Conference on Data Engineering (ICDE)

162

120

View full text Add to dashboard Cite

show abstract

“…Lopez et al [68] compare Apache Storm, Apache Flink, and Apache Spark Streaming in their paper. Besides describing the architecture of these three systems, the performance is studied in a network traffic analysis scenario.…”

Section: Related Workmentioning

confidence: 99%

DataEther: Data Exploration Framework For Ethereum

Chen

et al. 2019

2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS)

View full text Add to dashboard Cite

With the demand to process ever-growing data volumes, a variety of new data stream processing frameworks have been developed. Moving an implementation from one such system to another, e.g., for performance reasons, requires adapting existing applications to new interfaces. Apache Beam addresses these high substitution costs by providing an abstraction layer that enables executing programs on any of the supported streaming frameworks. In this paper, we present a novel benchmark architecture for comparing the performance impact of using Apache Beam on three streaming frameworks: Apache Spark Streaming, Apache Flink, and Apache Apex. We find significant performance penalties when using Apache Beam for application development in the surveyed systems. Overall, usage of Apache Beam for the examined streaming applications caused a high variance of query execution times with a slowdown of up to a factor of 58 compared to queries developed without the abstraction layer. All developed benchmark artifacts are publicly available to ensure reproducible results.

show abstract

“…Apache Spark has become increasingly popular in the last years also because it allows a developer to write her application in several different languages, without forcing her to think in terms of only map and reduce operators. Its good performance [48,49,50] and resilience to faults [51] has been empirically verified by various studies. We will first consider the case of large batches of input documents, and later refine the algorithm in order to accommodate streams as well.…”

Section: A Parallel Architecturementioning

confidence: 76%

Parallelizing Machine Learning as a service for the end-user

Loreti¹,

Lippi²,

Torroni³

2020

Future Generation Computer Systems

View full text Add to dashboard Cite

As Machine Learning (ML) applications are becoming ever more pervasive, fully-trained systems are made increasingly available to a wide public, allowing end-users to submit queries with their own data, and to efficiently retrieve results. With increasingly sophisticated such services, a new challenge is how to scale up to ever growing user bases. In this paper, we present a distributed architecture that could be exploited to parallelize a typical ML system pipeline. We propose a case study consisting of a text mining service, and discuss how the method can be generalized to many similar applications. We demonstrate the significance of the computational gain boosted by the distributed architecture by way of an extensive experimental evaluation.

show abstract

A Performance Comparison of Open-Source Stream Processing Platforms

Cited by 72 publications

References 16 publications

Benchmarking Distributed Stream Data Processing Systems

Benchmarking Distributed Stream Data Processing Systems

DataEther: Data Exploration Framework For Ethereum

Parallelizing Machine Learning as a service for the end-user

Contact Info

Product

Resources

About