Complex event processing over uncertain data

Abstract

In recent years, there has been a growing need for active systems that can react automatically to events. Some events are generated externally and deliver data across distributed systems, while others are materialized by the active system itself. Event materialization is hampered by uncertainty that may be attributed to unreliable data sources and networks, or the inability to determine with certainty whether an event has actually occurred. Two main obstacles exist when designing a solution to the problem of event materialization with uncertainty. First, event materialization should be performed efficiently, at times under a heavy load of incoming events from various sources. The second challenge involves the generation of a correct probability space, given uncertain events. We present a solution to both problems by introducing an efficient mechanism for event materialization under uncertainty. A model for representing materialized events is presented and two algorithms for correctly specifying the probability space of an event history are given. The first provides an accurate, albeit expensive method based on the construction of a Bayesian network. The second is a Monte Carlo sampling algorithm that heuristically assesses materialized event probabilities. We experimented with both the Bayesian network and the sampling algorithms, showing the latter to be scalable under an increasing rate of explicit event delivery and an increasing number of uncertain rules (while the former is not). Finally, our sampling algorithm accurately and efficiently estimates the probability space.

Towards anticipate detection of complex event processing rules with probabilistic modelling

“…Then, for each graph g, the prediction mechanism is launched by invoking the function make prediction (lines [3][4]. This function (lines 6-18) takes the current condition of the rule r cond state and uses its associated edge (E) to calculate the probability p of the derived event that will be generated by the rule according to its current state (lines 9-13).…”

“…However, the actual value of an event derived by an EPR activation decreases with time. In order to cope with this limitation, a prominent approach has made use of predictive analysis in order to forecast the occurrence of certain events in advance [4][5][6]. Nonetheless, this is achieved by means of external tools decoupled of the CEP system.…”

Streaming the Web: Reasoning over dynamic data

“…Efficiency in presence of uncertainty has been a main concern in stream processing systems and led to the design of promising techniques [125][126][127][128]123]. Finding a good balance between the advantages of a model in terms of accuracy, and simplicity, and the costs required for managing it constitutes a major area of investigation for future research.…”

Streaming the Web: Reasoning Over Dynamic Data

“…They include probability theory, Bayesian networks, fuzzy logics, and many other formalisms. The recent literature also presents proposals in the area of stream processing [125][126][127][128]123].…”