In recent years, there has been a growing need for active systems that can react automatically to events. Some events are generated externally and deliver data across distributed systems, while others are materialized by the active system itself. Event materialization is hampered by uncertainty that may be attributed to unreliable data sources and networks, or the inability to determine with certainty whether an event has actually occurred. Two main obstacles exist when designing a solution to the problem of event materialization with uncertainty. First, event materialization should be performed efficiently, at times under a heavy load of incoming events from various sources. The second challenge involves the generation of a correct probability space, given uncertain events. We present a solution to both problems by introducing an efficient mechanism for event materialization under uncertainty. A model for representing materialized events is presented and two algorithms for correctly specifying the probability space of an event history are given. The first provides an accurate, albeit expensive method based on the construction of a Bayesian network. The second is a Monte Carlo sampling algorithm that heuristically assesses materialized event probabilities. We experimented with both the Bayesian network and the sampling algorithms, showing the latter to be scalable under an increasing rate of explicit event delivery and an increasing number of uncertain rules (while the former is not). Finally, our sampling algorithm accurately and efficiently estimates the probability space.

No abstract

There is a growing need for the use of active systems, systems that act automatically based on events. In many cases, providing such active functionality requires materializing (inferring) the occurrence of relevant events. A widespread paradigm for enabling such materialization is Complex Event Processing (CEP), a rule based paradigm, which currently relies on domain experts to fully define the relevant rules. These experts need to provide the set of basic events which serves as input to the rule, their inter-relationships, and the parameters of the events for determining a new event materialization. While it is reasonable to expect that domain experts will be able to provide a partial rules specification, providing all the required details is a hard task, even for domain experts. Moreover, in many active systems, rules may change over time, due to the dynamic nature of the domain. Such changes complicate even further the specification task, as the expert must constantly update the rules. As a result, we seek additional support to the definition of rules, beyond expert opinion. This work presents a mechanism for automating both the initial definition of rules and the update of rules over time. This mechanism combines partial information provided by the domain expert with machine learning techniques, and is aimed at improving the accuracy of event specification and materialization. The proposed mechanism consists of two main repetitive stages, namely rule parameter prediction and rule parameter correction. The former is performed by updating the parameters using an available expert knowledge regarding the future changes of parameters. The latter stage utilizes expert feedback regarding the actual past occurrence of events and the events materialized by the CEP framework to tune rule parameters. We also include possible implementations for both stages, based on a statistical estimator and evaluate our outcome using a case study from the intrusion detection domain.

No abstract