Filtering algorithms and implementation for very fast publish/subscribe systems

Abstract

Publish/Subscribe is the paradigm in which users express long-term interests (“subscriptions”) and some agent “publishes” events (e.g., offers). The job of Publish/Subscribe software is to send events to the owners of subscriptions satisfied by those events. For example, a user subscription may consist of an interest in an airplane of a certain type, not to exceed a certain price. A published event may consist of an offer of an airplane with certain properties including price. Each subscription consists of a conjunction of (attribute, comparison operator, value) predicates. A subscription closely resembles a trigger in that it is a long-lived conditional query associated with an action (usually, informing the subscriber). However, it is less general than a trigger so novel data structures and implementations may enable the creation of more scalable, high performance publish/subscribe systems. This paper describes an attempt at the construction of such algorithms and its implementation. Using a combination of data structures, application-specific caching policies, and application-specific query processing our system can handle 600 events per second for a typical workload containing 6 million subscriptions.

Trust-based Revision for Expressive Web Syndication

“…Following this, there were efforts to capture more detailed user interests by allowing attribute-value pairs to be associated with published content [1,14,10,6,7,14]. In such an approach users are allowed to provide values for certain attributes, and matching reduces to determining the satisfaction of attribute values with associated publications.…”

Techniques for building a scalable and reliable distributed content-based publish/subscribe system

“…For the problem of content-match, various forms of decision trees and subscription indexes are proposed. These efforts can be largely clas-sified into two broad categories: the counting-based approach [12,19,40] and the tree-based approach [28,7,1].…”

“…Content-based event matching has been extensively studied in the literature. They can be mainly classified into two categories: counting algorithm [12,19,40] and tree-based approach [28,7,1,50]. We pick one indexing out of each category and implement them in our simulation.…”

eCube: Hypercube Event for Efficient Filtering in Content-Based Routing

“…Z-ordering interleaves the bits of each value for each dimension to create a one-dimensional bit string. For matching algorithms, fast and efficient matching algorithms are investigated for publish/subscribe system in [10]. Topic-based publish/subscribe is realised by a basic DHT-based multicast mechanism in [35], [36], [24].…”