Matching events in a content-based subscription system

Abstract

Content-based subscription systems are an emerging alternative to traditional publish-subscribe systems, because they permit more flexible subscriptions along multiple dimensions. In these systems, each subscription is a predicate which may test arbitrary attributes within an event. However, the matching problem for content-based systems -determining for each event the subset of all subscriptions whose predicates match the event -is still an open problem. We present an efficient, scalable solution to the matching problem. Our solution has an expected time complexity that is sub-linear in the number of subscriptions, and it has a space complexity that is linear. Specifically, we prove that for predicates reducible to conjunctions of elementary tests, the expected time to match a random event is no greater than ON 1, where N is the number of subscriptions, and is a closed-form expression that depends on the number and type of attributes (in some cases, 1=2). We present some optimizations to our algorithms that improve the search time. We also present the results of simulations that validate the theoretical bounds and that show acceptable performance levels for tens of thousands of subscriptions.

Contrail: Enabling Decentralized Social Networks on Smartphones

“…As can be observed from Figure 8a (label 'scan'), this approach very quickly results in a matching time of several seconds if the number of filters is large. In a second case we implemented a well known matching algorithm that uses a tree data structure to store the filters [4]. We generated filters in the worst possible manner which would cause the algorithm to visit every node in the tree while matching a data item.…”

MEDYM: Match-Early with Dynamic Multicast for Content-Based Publish-Subscribe Networks

“…In JEDI [9], a hierarchical event routing network was proposed, but was found to perform worse than the peer-to-peer topology in [6]. The Gryphon group [1][26] designed efficient content-based matching algorithms used in forwarding, and proposed using virtual time vectors to convey temporal consistency of subscription propagation. The Elvin system [21] proposed the concept of quenching, in which publishers are aware of the sum of all subscriptions in the system, so that they only publish events that have at least some interested subscribers.…”

“…Furthermore, note that the average destination list size in a dynamic multicast message, as analyzed in Section 3.1.3, is much shorter than the |DL| sizes in the table. Therefore, compared to the results on content-based matching [1][8], we expect the computation cost of dynamic multicast routing to be lower and the process faster. Route Caching.…”

The Architecture of the Cornell Knowledge Broker

“…It is functionally similar to a continuous data stream query processor [5,7,12,15,16], a publish/subscribe system [25], or an event processing system [3,13,14,38,44]. However, there are significant differences.…”