Analysis and optimization for boolean expression indexing

Abstract

BE-Tree is a novel dynamic data structure designed to efficiently index Boolean expressions over a high-dimensional discrete space. BE Tree-copes with both high-dimensionality and expressiveness of Boolean expressions by introducing an effective two-phase space-cutting technique that specifically utilizes the discrete and finite domain properties of the space. Furthermore, BE-Tree employs self-adjustment policies to dynamically adapt the tree as the workload changes. Moreover, in BE-Tree, we develop two novel cache-conscious predicate evaluation techniques, namely, lazy and bitmap evaluations, that also exploit the underlying discrete and finite space to substantially reduce BE-Tree's matching time by up to 75% BE-Tree is a general index structure for matching Boolean expression which has a wide range of applications including (complex) event processing, publish/subscribe matching, emerging applications in cospaces, profile matching for targeted web advertising, and approximate string matching. Finally, the superiority of BE-Tree is proven through a comprehensive evaluation with state-of-the-art index structures designed for matching Boolean expressions.

A Methodology for the Automatic Creation of Massive Continuous Query Datasets from Real Life Corpora

“…Some of these approaches include the systems XFilter [1], YFilter [6], DFA [10], the Boolean version of SIFT [17], and the agent-based DIAS [11]. Other approaches focus more on the algorithmic aspect by providing efficient treebased data structures such as [12,16,18,19,20] for dealing with documents that are free text and profiles that are conjunctions of keywords. To the best of our knowledge the only work that is somewhat relevant to ours is [15], where a corpus of documents (but no continuous queries) is built for adaptive filtering tasks.…”

“…To this end, a number of systems and algorithms that try to solve the filtering problem efficiently for different data models and query languages have been proposed [1,6,10,17,11,12,16,18,19,20]. However, despite all the research in the area, there is an apparent lack of a benchmarking mechanism (in the form of a large-scale standarised test collection of continuous queries and the relevant document publications) specifically created for evaluating filtering tasks.…”

“…Notice also that one-time queries, such as those obtained from public releases of major search engines' query logs (like Google BigQuery, Zeitgeist, or the AOL query set) are inappropriate for filtering tasks as they typically express a one-time information need, contrary to continuous queries that are used to express recurrent and long-standing information needs. Finally, other efforts, such as the TREC Filtering Track, are insufficient as they contain only a few dozens of manually created and curated continuous queries, and cannot live up to the need of modern benchmarking that is in the order of millions (e.g., as in [12,16,18,19,20]). …”

An efficient publish/subscribe index for e-commerce databases

“…Publish/subscribe, or pub/sub for brevity, has been well-studied in the last two decades [3,6,9,16,20,22,26], with deployment in a variety of applications including online advertising [16], stock market [6] and social media monitoring [9]. A pub/sub system contains two types of roles, information provider and information consumer.…”

“…The information consumer subscribes interesting events in the form of boolean expression. These two roles can be interconnected either via a simple client/server model [12,20,22,26] or over a network of brokers routing events in a distributed paradigm [3,7,14]. The system has to ensure a timely delivery of matching events to the subscribers.…”