Predictable Low-Latency Event Detection With Parallel Complex Event Processing

Mayer, Ruben; Koldehofe, Boris; Rothermel, Kurt

doi:10.1109/jiot.2015.2397316

Cited by 66 publications

(55 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…eir goal is to minimize a latency-violation penalty, energy-consumption cost and consider a system stability value to avoid oscillation. Similar to the work of Mayer et al [96], a disturbance forecaster predicts the future arrival rate and processing time. As in Lohrmann et al [90], De Ma eis and Mencagli predict the average per-tuple latency with QT.…”

Section: Centralized Elasticitymentioning

confidence: 96%

“…e window slide de nes the intervals the operator starts a new window on the input stream. It, too, can be time-or count based or rely on a predicate [8,56,96]. For a deeper discussion of windows and classi cation of windowing policies, we point to the literature [50,70].…”

Section: Programmingmentioning

confidence: 99%

“…Single Machine x Fernandez [45] GP Imp External Cloud x x Lohrmann [91] GP Imp Cluster, Cloud x x Zygouras [158] CEP Dec Cloud x Schneider [125] GP Dec Cloud x x Rive i [118] GP Imp Cluster x Mayer [96,99] CEP Imp Cloud x Wu [147] GP Imp (External) Cluster x Nasir [108,109] GP Imp Cluster x x Saleh [120] CEP Dec Cluster x x Koliousis [78] GP Dec GPU x x x Zacheilas [153] CEP Dec Cloud x Nakamura [107] GP Imp Fog x Gedik [53] GP Spark Streaming and Structured Streaming by Spark [154]: As Spark applications originally processed batches, the Spark Streaming extensions process streamed data in micro batches. Structured Streaming interprets data streams as an unbounded table where each new data item extends the table.…”

Section: Parallelizationmentioning

confidence: 99%

See 2 more Smart Citations

A Comprehensive Survey on Parallelization and Elasticity in Stream Processing

Röger

Mayer

2019

ACM Comput. Surv.

Self Cite

View full text Add to dashboard Cite

Stream Processing (SP) has evolved as the leading paradigm to process and gain value from the high volume of streaming data produced e.g. in the domain of the Internet of ings. An SP system is a middleware that deploys a network of operators between data sources, such as sensors, and the consuming applications. SP systems typically face intense and highly dynamic data streams. Parallelization and elasticity enables SP systems to process these streams with continuously high quality of service. e current research landscape provides a broad spectrum of methods for parallelization and elasticity in SP. Each method makes speci c assumptions and focuses on particular aspects of the problem. However, the literature lacks a comprehensive overview and categorization of the state of the art in SP parallelization and elasticity, which is necessary to consolidate the state of the research and to plan future research directions on this basis. erefore, in this survey, we study the literature and develop a classi cation of current methods for both parallelization and elasticity in SP systems. or even di erent processing nodes in a shared-nothing cloud-based infrastructure. Frequent state synchronization must not hamper parallel processing, while the processing results have to remain consistent. Research proposes di erent approaches for parallel, stateless and stateful SP. ey di er in assumptions about the operator functions and state externalization mechanisms an SP system supports.is led to the development of a broad range of parallelization approaches tackling di erent problem cases.Second, how to continuously adapt the level of parallelization when the conditions of the SP operators, e.g. the workload or resources available, change at runtime. On the one hand, an SP system always needs enough resources to process the input data streams with a satisfying quality of service (QoS), e.g. latency or throughput. On the other hand, continuous provisioning of computing resources for peak workloads wastes resources at o -peak hours. us, an elastic SP system scales its resources according to the current need. Cloud computing provides on-demand resources to realize such elasticity [9]. e pay-as-you-go business model of cloud computing allows to cut costs by dynamically adapting the resource reservations to the needs of the SP system. It is challenging to strive the right balance between resource over-provisioning-which is costly, but is robust to workload uctuations-and on-demand scaling-which is cheap, but is vulnerable to sudden workload peaks. To this end, academia and industry developed elasticity methods. Again, they di er in their optimization objectives and assumptions about the operator parallelization model employed, the target system architecture, state management as well as timing and methodology.While there are many works that propose methods and solutions for speci c parallelization and elasticity problems in SP systems, there is a severe lack of overview, comparison, and classi cation of these methods. When we investigate...

show abstract

Section: Centralized Elasticitymentioning

confidence: 96%

Section: Programmingmentioning

confidence: 99%

Section: Parallelizationmentioning

confidence: 99%

See 1 more Smart Citation

A Comprehensive Survey on Parallelization and Elasticity in Stream Processing

Röger

Mayer

2019

ACM Comput. Surv.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Provenance Recording for Services (PReServ) is a web service implementation of the PReP protocol that stores the provenance either in memory, in a relational database, or in the file system 4) PUFs and wireless link finger prints A PUF is a physically disordered system that maps a set of challenges to a set of responses based on the underlying physical micro structure of the device and it is very difficult to clone and wireless finger print in any of the wireless channel parameters such as the received signal strength indicator (RSSI) may uniquely identify a wireless link between two parties . the received signal strength indicator (RSSI) [16]values of a wireless channel can be used as the fingerprint of a wireless link. However, their technique requires the device to store a secret key and depends on the public key infrastructure IV.…”

Section: ) Provenance Aware Service-oriented Architecturementioning

confidence: 99%

Security & privacy in IoT Data Provenance

T.K¹,

Jebakumar²

2018

IJET

View full text Add to dashboard Cite

Abstract-IoT has become most emerging trend in engineering and technology, that has bring relevant change in industries. IoT has many applications like healthcare, agriculture etc. Since IoT deals with huge number of objects that need to generate and consume different types of data and entity. The need of security and privacy challenges should enforced. The data provenance deals with detecting creation and propagation of data, The main purpose of the paper is to address security and privacy issues by implementing data provenance in IoT. To this end we will mention different challenges to meet security and privacy in IoT data provenance

show abstract

“…With the need for high performance, distributed CEP systems are developed by distributing the detection logic over a network of operators, where individual operators can be a bottleneck, requiring operator parallelization. 42 As far as we know, only a few recent works focus on parallelization of pattern-matching processing in which pattern matching is processed as a stateful operator in a generalpurpose streaming system. Hirzel 30 exploits the partitioning constructs provided by the queries, such as PARTITION BY.…”

Section: Related Workmentioning

confidence: 99%

New parallel processing strategies in complex event processing systems with data streams

Xiao

Zhan

Lai

et al. 2017

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

Sensor network-based application has gained increasing attention where data streams gathered from distributed sensors need to be processed and analyzed with timely responses. Distributed complex event processing is an effective technology to handle these data streams by matching of incoming events to persistent pattern queries. Therefore, a well-managed parallel processing scheme is required to improve both system performance and the quality-of-service guarantees of the system. However, the specific properties of pattern operators increase the difficulties of implementing parallel processing. To address this issue, a new parallelization model and three parallel processing strategies are proposed for distributed complex event processing systems. The effects of temporal constraints, for example, sliding windows, are included in the new parallelization model to enable the processing load for the overlap between windows of a batch induced by each input event to be shared by the downstream machines to avoid events that may result in wrong decisions. The proposed parallel strategies can keep the complex event processing system working stably and continuously during the elapsed time. Finally, the application of our work is demonstrated using experiments on the StreamBase system regardless of the increased input rate of the stream or the increased time window size of the operator.

show abstract

Predictable Low-Latency Event Detection With Parallel Complex Event Processing

Cited by 66 publications

References 42 publications

A Comprehensive Survey on Parallelization and Elasticity in Stream Processing

A Comprehensive Survey on Parallelization and Elasticity in Stream Processing

Security & privacy in IoT Data Provenance

New parallel processing strategies in complex event processing systems with data streams

Contact Info

Product

Resources

About