MavEStream: Synergistic Integration of Stream and Event Processing

Jiang, Qingchun; Adaikkalavan, Raman; Chakravarthy, Sharma

doi:10.1109/icdt.2007.21

Cited by 15 publications

(7 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jiang et al combined event and stream-data processing in MavEStream [17]. They use continuous queries to generate computed events from data streams, then cascade them into an event-processing engine for complex events.…”

Section: Pattern Matchingmentioning

confidence: 99%

An algebra for pattern matching, time-aware aggregates and partitions on relational data streams

Herbst

Pollner

Tenschert

et al. 2015

Proceedings of the 9th ACM International Conference on Distributed Event-Based Systems

View full text Add to dashboard Cite

Many interesting applications of continuous-query processing are concerned with pattern matching or complex temporal aggregation of events. Real-world queries that rely on these operations are difficult to implement in current streamprocessing systems. The reason seems to be a gap between two types of existing query languages: Some languages (e. g. CQL) offer a small set of simple operators that can be combined in order to create complex queries. While these languages provide sound and comprehensible semantics, they lack the expressiveness required for many real-world applications. Other approaches (e. g. Aurora) provide powerful operators but lack semantic strictness, which is required for reasoning about query results. Such reasoning is a prerequisite for safe query optimization.We try to bridge this gap by integrating operators for pattern matching and time-aware aggregates into a generalpurpose stream model featuring stream partitioning. These operators can answer several questions that we have found to be relevant in a real-world object-tracking scenario. Moreover, they are formally defined, allowing expressive and efficient queries to be written in CQL-like languages, while remaining understandable and easy to use.

show abstract

Section: Pattern Matchingmentioning

confidence: 99%

An algebra for pattern matching, time-aware aggregates and partitions on relational data streams

Herbst

Pollner

Tenschert

et al. 2015

Proceedings of the 9th ACM International Conference on Distributed Event-Based Systems

View full text Add to dashboard Cite

show abstract

“…Recent work explores the synergy between stream processing and event processing [9,17]. A number of different load distribution techniques have been studied in the context of distributed stream processing.…”

Section: Related Workmentioning

confidence: 99%

A stratified approach for supporting high throughput event processing applications

Lakshmanan¹,

Rabinovich

Etzion

2009

Proceedings of the Third ACM International Conference on Distributed Event-Based Systems

View full text Add to dashboard Cite

The quantity of events that a single application needs to process is constantly increasing. RFID related events have doubled within the past year and reached 4 trillion events per day, financial applications in large banks are processing 400 million events per day, and Massively Multiplayer Online (MMO) games are monitoring millions of events per second during peak periods. It is evident that scalability in event throughput is a major requirement for such applications. While the first generation of event processing systems is centralized, we see various solutions that attempt to use both scale-up and scale-out techniques. Alas, partitioning of the processing manually is difficult due to the semantic dependencies among event processing agents. It is also difficult to manually tune up the partition dynamically. This paper proposes a horizontal partition that is automatically created by analyzing the semantic dependencies among agents using a stratification principle. Each stratum contains a collection of independent agents, and events are routed to subsequent strata. We also implement a profiling-based technique for assigning agents to nodes in each stratum with the goal of maximizing throughput. A complementary step is to distribute load among different execution nodes dynamically based on their performance characteristics and the event traffic model. Experimental results show significant improvement in the ability to process high throughput of events relative to both centralized solutions as well as vertical partitions. We find this to be a promising approach to achieve high scalability particularly when the traffic model and network topology change frequently.

show abstract

“…The event generation stage was added to combine the two relatively easily. Of course, a number of additional issues still need to be addressed in the above integration which can be found in [86,105,109,110].…”

Section: Integration Of Stream and Event Processingmentioning

confidence: 99%

Events and streams

Chakravarthy

Adaikkalavan

2008

Proceedings of the Second International Conference on Distributed Event-Based Systems

Self Cite

View full text Add to dashboard Cite

One of the purposes of this paper is to demonstrate the ubiquitous nature and relevance of simple and complex events in situation monitoring and other unforeseen applications. This paper retrospectively analyzes the progress of eventbased capability and their applicability in various domains. Although research on event-based approaches started in a humble manner with the intention of introducing triggers in database management systems for monitoring application state and to automate applications by reducing/eliminating user intervention, currently it has become a force to reckon with as it finds use in many diverse domains. This is primarily due to the fact that a large number of real-world applications are indeed event-driven and hence the paradigm is apposite.The other purpose of this paper is to identify and establish the synergy between stream processing and event processing. The resurgence of complex event processing (or CEP) has demonstrated the resiliency of this paradigm and its relevance to a large class of applications. In addition, the advent of stream processing based on sensor and other data generated on a continuous basis has, in our opinion, enhanced the role of events in critical ways. Instead of implicitly assuming event generation, stream processing indeed provides a powerful mechanism for generating interesting events.In this paper, we briefly overview the development of the ECA (or event-condition-action) paradigm. We briefly discuss the evolution of the ECA paradigm (or active capability) in Relational and Object-oriented systems. We then describe several diverse applications where the ECA paradigm has been used effectively. The applications range from customized monitoring of web pages to specification and enforcement of role-based access control policies (RBAC). The multitude of applications clearly demonstrate the ubiquitous

show abstract

MavEStream: Synergistic Integration of Stream and Event Processing

Cited by 15 publications

References 22 publications

An algebra for pattern matching, time-aware aggregates and partitions on relational data streams

An algebra for pattern matching, time-aware aggregates and partitions on relational data streams

A stratified approach for supporting high throughput event processing applications

Events and streams

Contact Info

Product

Resources

About