A combination of trie-trees and inverted files for the indexing of set-valued attributes

Abstract

Set-valued attributes frequently occur in contexts like marketbasked analysis and stock market trends. Late research literature has mainly focused on set containment joins and data mining without considering simple queries on set valued attributes. In this paper we address superset, subset and equality queries and we propose a novel indexing scheme for answering them on set-valued attributes. The proposed index superimposes a trie-tree on top of an inverted file that indexes a relation with set-valued data. We show that we can efficiently answer the aforementioned queries by indexing only a subset of the most frequent of the items that occur in the indexed relation. Finally, we show through extensive experiments that our approach outperforms the state of the art mechanisms and scales gracefully as database size grows.

Comprehensive Study and Comparison of Information Retrieval Indexing Techniques

“…There is no necessity of associating values with every node. Values are instead associated with leaves alongside several inner nodes that have correspondence to keys of interest [7]. Fig.…”

Index Data Structure for Fast Subset and Superset Queries

“…Secondly, the subset queries are studied in various sub-areas of AI for storing and querying: pre-conditions of a large set of rules [6], states in planning for storing goal sets [8] and hypotheses in data mining algorithms [9]. Finally, querying sets is an important problem in object-relational databases management systems where attributes of relations can range over sets [18,12,7,19,20].…”

“…Set-valued attributes are used for the representation of properties that range over sets of atomic values or objects. Database community has shown significant interest in indexing structures that can be used as access paths for querying set-valued attributes [18,12,7,19,20].…”

“…Set containment queries can express either selection or join operation based on set containment condition. Efficient access to relation records based on conditions that involve set operations are vital for fast implementation of such queries [12,18,7].…”

Efficient answering of set containment queries for skewed item distributions

“…Having lists that vary substantially in size, is a result of skewed item distributions in the database, which is a common real world case. The Hybrid Trie Inverted file (HTI) [42], breaks up the larger inverted lists to smaller sublists that contain known combinations of items. This approach provides superior performance over the inverted file for skewed distributions (a comparison with OIF appears in [40]).…”