Wihidum: Distributed complex event processing

Jayasekara, Sachini; Kannangara, Sameera; Dahanayakage, Tishan; Ranawaka, Isuru; Perera, Srinath; Nanayakkara, Vishaka

doi:10.1016/j.jpdc.2015.03.002

Cited by 21 publications

(12 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A CEPE supports exploring relationships among events using specific rules as well as declarative interfaces [8]. Cutting edge CEPEs include Esper, StreamBase, and WSO2 CEP [30], [34], [36]. Generic-purpose DSPEs (GDSPE) are being developed to perform distributed stream processing with the aim of achieving scalable and fault-tolerant execution on cluster environments.…”

Section: ) Data Stream Processing Layermentioning

confidence: 99%

A Survey of Distributed Data Stream Processing Frameworks

et al. 2019

View full text Add to dashboard Cite

Big data processing systems are evolving to be more stream oriented where each data record is processed as it arrives by distributed and low-latency computational frameworks on a continuous basis. As the stream processing technology matures and more organizations invest in digital transformations, new applications of stream analytics will be identified and implemented across a wide spectrum of industries. One of the challenges in developing a streaming analytics infrastructure is the difficulty in selecting the right stream processing framework for the different use cases. With a view to addressing this issue, in this paper we present a taxonomy, a comparative study of distributed data stream processing and analytics frameworks, and a critical review of representative open source (Storm, Spark Streaming, Flink, Kafka Streams) and commercial (IBM Streams) distributed data stream processing frameworks. The study also reports our ongoing study on a multilevel streaming analytics architecture that can serve as a guide for organizations and individuals planning to implement a real-time data stream processing and analytics framework. INDEX TERMS Dataflow architectures, data stream architectures, distributed processing systems comparison, survey, taxonomy.

show abstract

Section: ) Data Stream Processing Layermentioning

confidence: 99%

A Survey of Distributed Data Stream Processing Frameworks

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Real-time Query Engine: The real-time query engine is built around an existing open-source CEP engine, WSO2 Siddhi [35,21]. Siddhi is an exemplar of a traditional CEP engine and uses a tuple-based event model to support common CEP operators for performing filtering, sequence and aggregation patterns over event streams.…”

Section: Scepter System Architecturementioning

confidence: 99%

Knowledge-infused and consistent Complex Event Processing over real-time and persistent streams

Zhou

Simmhan

Prasanna

2017

Future Generation Computer Systems

View full text Add to dashboard Cite

Emerging applications in Internet of Things (IoT) and Cyber-Physical Systems (CPS) present novel challenges to Big Data platforms for performing online analytics. Ubiquitous sensors from IoT deployments are able to generate data streams at high velocity, that include information from a variety of domains, and accumulate to large volumes on disk. Complex Event Processing (CEP) is recognized as an important real-time computing paradigm for analyzing continuous data streams. However, existing work on CEP is largely limited to relational query processing, exposing two distinctive gaps for query specification and execution: (1) infusing the relational query model with higher level knowledge semantics, and (2) seamless query evaluation across temporal spaces that span past, present and future events. These allow accessible analytics over data streams having properties from different disciplines, and help span the velocity (real-time) and volume (persistent) dimensions. In this article, we introduce a Knowledge-infused CEP ( χ -CEP) framework that provides domain-aware knowledge query constructs along with temporal operators that allow end-to-end queries to span across real-time and persistent streams. We translate this query model to efficient query execution over online and offline data streams, proposing several optimizations to mitigate the overheads introduced by evaluating semantic predicates and in accessing high-volume historic data streams. In particular, we also address temporal consistency issues that arise during fault recovery of query plans that span the boundary between real-time and persistent streams. The proposed χ -CEP query model and execution approaches are implemented in our prototype semantic CEP engine, SCEPter. We validate our query model using domain-aware CEP queries from a real-world Smart Power Grid application, and experimentally analyze the benefits of our optimizations for executing these queries, using event streams from a campus-microgrid IoT deployment. Our results show that we are able to sustain a processing throughput of 3, 000 events/secs for χ -CEP queries, a 30× improvement over the baseline and sufficient to support a Smart Township, and can resume consistent processing within 20 secs after stream outages as long as 2 hours.

show abstract

“…In our engine, we distinguish atomic events (ae) and complex events (ce), it is similar with the definition in [18]- [20]. Atomic events are single events which occur at a particular point of time.…”

Section: Event Algebramentioning

confidence: 99%

A Novel Engine: Complex Event Processing with LAIPE

Wang¹

2017

JCP

View full text Add to dashboard Cite

Most of the Complex Event Processing (CEP) engines are responsible for filtering, selecting and aggregating atomic events to represent high level composite events according to predefined rules are the latest breed of event-oriented software, which is being presented nowadays to better support timely information processing. Introducing CEP engines to the information system domain provides an opportunity to enhance the capabilities of event processing in real-time. To answer this need, we first propose an event algebra that defines the detailed semantics of event models and operators. More specifically, we propose a Lightweight Automata based Incremental Processing Engine (LAIPE) based on a layered architectural design. On the one hand, it adopts a language (LAIPL) and its rule structure which explicitly conceived to natural and easy identification of complex events. On the other hand, it provides efficient algorithms for rule translation and event incremental processing approach based on event automaton model. Finally, our evaluation on LAIPE and existing Esper engine shows that the time cost of LAIPE is averagely reduced 42.3% than Esper in different tests, and our developed engine has better performance in processing different sliding window and selectivity rates with a large number of events.

show abstract

Wihidum: Distributed complex event processing

Cited by 21 publications

References 4 publications

A Survey of Distributed Data Stream Processing Frameworks

A Survey of Distributed Data Stream Processing Frameworks

Knowledge-infused and consistent Complex Event Processing over real-time and persistent streams

A Novel Engine: Complex Event Processing with LAIPE

Contact Info

Product

Resources

About