A framework for performance evaluation of complex event processing systems

Mendes, Marcelo R.N.; Bizarro, Pedro; Marques, Paulo

doi:10.1145/1385989.1386030

Cited by 25 publications

(12 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to the rapid growth of event processing technology, large variety of application domains, and lack of standards [13], performance metrics are subject to various interpretations, often leading to incomparable product benchmarks. Among the most commonly used performance indicators are throughput and latency (response time), as well as scalability, security, correctness and other non-functional requirements.…”

Section: Pattern Performance Objectivesmentioning

confidence: 99%

“…We use events per second (event/s) to measure throughput. Similarly to [2] and [13], we define a system latency as a delay between the last input event causing a certain scenario detection and the detection itself, resulting in derivation of an output event. An application's latency is usually measured in milliseconds (ms).…”

Section: Pattern Performance Objectivesmentioning

confidence: 99%

See 1 more Smart Citation

Pattern rewriting framework for event processing optimization

Rabinovich

Etzion

Gal

2011

Proceedings of the 5th ACM International Conference on Distributed Event-Based System

View full text Add to dashboard Cite

A growing segment of event-based applications require both strict performance goals and support in the processing of complex event patterns. Event processing patterns have multiple complexity dimensions: the semantics of the language constructs (e.g., sequence) and the variety of semantic interpretations for each pattern (controlled by policies). We introduce in this paper a novel approach for pattern rewriting that aims at efficiently processing patterns which comprise all levels of complexity. We present a formal model for pattern rewriting and demonstrate its usage in a comprehensive set of rewriting techniques for complex pattern types, taking various semantic interpretations into account. A cost model is presented, balancing processing latency and event throughput according to user's preference. Pattern cost is then estimated using simulation-based techniques.This work advances the state-of-the-art by analyzing complex event processing logic and by using explicit means to optimize elements that were considered "black box." Our empirical study yields encouraging results, with improvement gain of up to tenfold relative to the non optimized solutions that are used in the current state-of-the-art systems.

show abstract

Section: Pattern Performance Objectivesmentioning

confidence: 99%

Section: Pattern Performance Objectivesmentioning

confidence: 99%

Pattern rewriting framework for event processing optimization

Rabinovich

Etzion

Gal

2011

Proceedings of the 5th ACM International Conference on Distributed Event-Based System

View full text Add to dashboard Cite

show abstract

“…Several test harnesses and benchmarks for different EBS were published, e.g., [8,5,7]. However, previous work in the area of benchmarking mostly focuses on the design and development of test frameworks, but not on the definition of workloads.…”

Section: Related Workmentioning

confidence: 99%

A Logistics Workload for Event Notification Middleware

Appel

Sachs

2010

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. The event-based paradigm plays an important role to reflect logistics processes in modern IT infrastructures. Events occur at many stages, e.g., when goods tagged with RFID chips are scanned, when transportation vehicles move or when sensors report environmental observations. These events have to be delivered to interested consumers by a reliable notification middleware, which is crucial for a successful implementation of event-based applications. Specified service levels have to be fulfilled and to guarantee them, an exhaustive evaluation and analysis of the underlying event notification middleware is required. This can be achieved by applying well-defined test scenarios that allow us to analyze different aspects of the middleware in an independent and representative way. In this paper we present a realistic workload originating from a real world scenario in the logistics domain. Our workload is suited to test event notification middleware under realistic conditions; it stresses different aspects of the middleware while being scalable.

show abstract

“…The input streams data were generated and submitted using the FINCoS framework [15], a set of benchmarking tools we have developed for assessing performance of CEP engines. Both the load generation components and the event processing engines under test ran in a single machine to eliminate network latencies and jitter.…”

Section: Tests Setupmentioning

confidence: 99%

A Performance Study of Event Processing Systems

Mendes

Bizarro

Marques

2009

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. Event processing engines are used in diverse mission-critical scenarios such as fraud detection, traffic monitoring, or intensive care units. However, these scenarios have very different operational requirements in terms of, e.g., types of events, queries/patterns complexity, throughput, latency and number of sources and sinks. What are the performance bottlenecks? Will performance degrade gracefully with increasing loads? In this paper we make a first attempt to answer these questions by running several micro-benchmarks on three different engines, while we vary query parameters like window size, window expiration type, predicate selectivity, and data values. We also perform some experiments to assess engines scalability with respect to number of queries and propose ways for evaluating their ability in adapting to changes in load conditions. Lastly, we show that similar queries have widely different performances on the same or different engines and that no engine dominates the other two in all scenarios.

show abstract

A framework for performance evaluation of complex event processing systems

Cited by 25 publications

References 4 publications

Pattern rewriting framework for event processing optimization

Pattern rewriting framework for event processing optimization

A Logistics Workload for Event Notification Middleware

A Performance Study of Event Processing Systems

Contact Info

Product

Resources

About