Data structure set-trie for storing and querying sets: Theoretical and empirical analysis

Abstract

Set containment operations form an important tool in various fields such as information retrieval, AI systems, object-relational databases, and Internet applications. In the paper, a set-trie data structure for storing sets is considered, along with the efficient algorithms for the corresponding set containment operations. We present the mathematical and empirical study of the set-trie. In the mathematical study, the relevant upper-bounds on the efficiency of its expected performance are established by utilizing a natural probabilistic model. In the empirical study, we give insight into how different distributions of input data impact the efficiency of set-trie. Using the correct parameters for those randomly generated datasets, we expose the key sources of the input sensitivity of set-trie. Finally, the empirical comparison of set-trie with the inverted index is based on the real-world datasets containing sets of low cardinality. The comparison shows that the running time of set-trie consistently outperforms the inverted index by orders of magnitude.

Group Tracking for Video Monitoring Systems: A Spatio-Temporal Query Processing Approach

“…Our GTR maintenance includes a lot of set operations which are checking the existence of groups, and searching the subsets of groups. Therefore, we can apply a set-trie index data structure [21], [22] to accelerate GTR maintenance. Set-trie is a tree composed of nodes labeled with an element in the domain of [1,N].…”

Encapsulation structure and dynamics in hypergraphs

“…We show that the worst-case time complexity is bounded by the product of the number of hyperedges m, the largest hyperedge size s max , and the largest node degree k max . We also discuss and informally evaluate an alternative strategy using the set-trie data structure [54].…”

“…Therefore, the above running time is an upper-bound on the most efficient algorithm. One potentially more efficient alternative is to construct an encapsulation DAG by augmenting the construction of a set-trie data structure [54]. Here we give an informal description of one implementation of this strategy and its complexity.…”

“…< u se ) sorted in increasing order by node label. The algorithm constructs a set-trie by looping over the hyperedges e in increasing order of size, first running the GetAllSubsets procedure specified in [54] on e, then inserting e into the set-trie. Since we are inserting the hyperedges in order of their size, we can be sure that all possible subsets of e already exist in the data structure before e is inserted, thus GetAllSubsets returns all DAG edges for each hyperedge e.…”

“…where q is an assumed fixed probability of a hyperedge to be excluded from a set of hyperedges over V, i.e. the probability of inclusion in the hypergraph is assumed to be P[e ∈ E] = p and q = 1 − p (see [54] sections 4.2 and 4.3.1 for details). Thus an informal bound can be stated as O(m(s max + E(we))).…”