Scalable Pattern Sharing on Event Streams*

Ray, Medhabi; Lei, Chuan; Rundensteiner, Elke A.

doi:10.1145/2882903.2882947

Cited by 41 publications

(37 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the time window, we use the event's source timestamps. We measured a common metric [8,25,52,63] for streaming system, namely average CPU time over the windows. Average CPU time is measured in seconds and milliseconds as the sum of total elapsed time in all windows divided by the number of windows.…”

Section: Methodsmentioning

confidence: 99%

A query language for semantic complex event processing: Syntax, semantics and implementation

Gillani

Zimmermann

Picard

et al. 2018

View full text Add to dashboard Cite

The field of Complex Event Processing (CEP) relates to the techniques and tools developed to efficiently process pattern-based queries over data streams. The Semantic Web, through its standards and technologies, is in constant pursue to provide solutions for such paradigm while employing the RDF data model. The integration of Semantic Web technologies in this context can handle the heterogeneity, integration and interpretation of data streams at semantic level. In this paper, we propose and implement a new query language, called SPA , that extends SPARQL with new Semantic Complex Event Processing (SCEP) operators that can be evaluated over RDF graph-based events. The novelties of SPA includes (i) the separation of general graph pattern matching constructs and temporal operators; (ii) the support for RDF graph-based events and multiple RDF graph streams; and (iii) the expressibility of temporal operators such as Kleene+, conjunction, disjunction and event selection strategies; and (iv) the operators to integrate background information and streaming RDF graph streams. Hence, SPA enjoys good expressiveness compared with the existing solutions. Furthermore, we provide an efficient implementation of SPA using a non-deterministic automata (NFA) model for an efficient evaluation of the SPA queries. We provide the syntax and semantics of SPA and based on this, we show how it can be implemented in an efficient manner. Moreover, we also present an experimental evaluation of its performance, showing that it improves over state-of-the-art approaches.

show abstract

Section: Methodsmentioning

confidence: 99%

A query language for semantic complex event processing: Syntax, semantics and implementation

Gillani

Zimmermann

Picard

et al. 2018

View full text Add to dashboard Cite

show abstract

“…As a result, redundant computations will be performed by automata or trees corresponding to different subpatterns (comparing A's to B's in our example). This problem can be solved by applying known multi-query techniques for shared subexpression processing, such as those described in [17,34,42,43,53].…”

Section: Nested Patternsmentioning

confidence: 99%

“…As already shown above, since the leaves of an evaluation tree cannot be reordered, it only searches through a partial solution space. Advanced methods were also proposed for multi-query CEP optimization [17,34,42,43,53].…”

Section: Related Workmentioning

confidence: 99%

Join query optimization techniques for complex event processing applications

Kolchinsky

Schuster

2018

Proc. VLDB Endow.

View full text Add to dashboard Cite

Complex event processing (CEP) is a prominent technology used in many modern applications for monitoring and tracking events of interest in massive data streams. CEP engines inspect real-time information flows and attempt to detect combinations of occurrences matching predefined patterns. This is done by combining basic data items, also called "primitive events", according to a pattern detection plan, in a manner similar to the execution of multi-join queries in traditional data management systems. Despite this similarity, little work has been done on utilizing existing join optimization methods to improve the performance of CEP-based systems.In this paper, we provide the first theoretical and experimental study of the relationship between these two research areas. We formally prove that the CEP Plan Generation problem is equivalent to the Join Query Plan Generation problem for a restricted class of patterns and can be reduced to it for a considerably wider range of classes. This result implies the NP-completeness of the CEP Plan Generation problem. We further show how join query optimization techniques developed over the last decades can be adapted and utilized to provide practically efficient solutions for complex event detection. Our experiments demonstrate the superiority of these techniques over existing strategies for CEP optimization in terms of throughput, latency, and memory consumption.

show abstract

“…Complete Event Trend. We call the matches of a CET query event trends instead of using the traditional term of an event sequence since event sequences usually have fixed, statically known length [23,30,31].…”

Section: Cet Data and Query Modelmentioning

confidence: 99%

Complete Event Trend Detection in High-Rate Event Streams

Poppe

Lei

Ahmed

et al. 2017

Proceedings of the 2017 ACM International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

Event processing applications from financial fraud detection to health care analytics continuously execute event queries with Kleene closure to extract event sequences of arbitrary, statically unknown length, called Complete Event Trends (CETs). Due to common event sub-sequences in CETs, either the responsiveness is delayed by repeated computations or an exorbitant amount of memory is required to store partial results. To overcome these limitations, we define the CET graph to compactly encode all CETs matched by a query. Based on the graph, we define the spectrum of CET detection algorithms from CPU-optimal to memoryoptimal. We find the middle ground between these two extremes by partitioning the graph into time-centric graphlets and caching partial CETs per graphlet to enable effective reuse of these intermediate results. We reveal cost monotonicity properties of the search space of graph partitioning plans. Our CET optimizer leverages these properties to prune significant portions of the search to produce a partitioning plan with minimal CPU costs yet within the given memory limit. Our experimental study demonstrates that our CET detection solution achieves up to 42-fold speedup even under rigid memory constraints compared to the state-of-the-art techniques in diverse scenarios.

show abstract

Scalable Pattern Sharing on Event Streams*

Cited by 41 publications

References 40 publications

A query language for semantic complex event processing: Syntax, semantics and implementation

A query language for semantic complex event processing: Syntax, semantics and implementation

Join query optimization techniques for complex event processing applications

Complete Event Trend Detection in High-Rate Event Streams

Contact Info

Product

Resources

About