Efficient processing of probabilistic set-containment queries on uncertain set-valued data

Zhang, Xiaolong; Chen, Ke; Shou, Lidan; Chen, Gang; Yuan, Guandou; Tan, Kian-Lee

doi:10.1016/j.ins.2012.02.004

Cited by 5 publications

(7 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(4) In practice, both queries and data are often noisy and uncertain. It would be interesting to investigate extensions to handle query relaxations such as set similarity joins and set containment in uncertain data models [1,23,31,39]. (5) More sophisticated pruning mechanisms for hierarchical data could be profitably developed for our context, e.g., based on recent work such as [16,33].…”

Section: Discussionmentioning

confidence: 99%

Efficient processing of containment queries on nested sets

Ibrahim

Fletcher

2013

Proceedings of the 16th International Conference on Extending Database Technology

View full text Add to dashboard Cite

We study the problem of computing containment queries on sets which can have both atomic and set-valued objects as elements, i.e., nested sets. Containment is a fundamental query pattern with many basic applications. Our study of nested set containment is motivated by the ubiquity of nested data in practice, e.g., in XML and JSON data management, in business and scientific workflow management, and in web analytics. Furthermore, there are to our knowledge no known efficient solutions to computing containment queries on massive collections of nested sets. Our specific contributions in this paper are: (1) we introduce two novel algorithms for efficient evaluation of containment queries on massive collections of nested sets; (2) we study caching and filtering mechanisms to accelerate query processing in the algorithms; (3) we develop extensions to the algorithms to a) compute several related query types and b) accommodate natural variations of the semantics of containment; and, (4) we present analytic and empirical analyses which demonstrate that both algorithms are efficient and scalable.

show abstract

Section: Discussionmentioning

confidence: 99%

Efficient processing of containment queries on nested sets

Ibrahim

Fletcher

2013

Proceedings of the 16th International Conference on Extending Database Technology

View full text Add to dashboard Cite

show abstract

“…[27] Most The closest work to ours is [39], [23], [27]. [39] considers efficient query processing methods for set containment queries for probabilistic sets. Their probabilistic set model is identical to ours.…”

Section: Related Workmentioning

confidence: 99%

“…One of the similarity measure considered (expected Jaccard containment) and its dynamic programming-based computation algorithm are similar to our expected Jaccard similarity and its computation algorithm. However, [39] does not consider pruning method for individual sets or batch pruning, nor do they consider approximate query processing methods, as the probabilistic sets considered contain small number of elements (up to 25).…”

Section: Related Workmentioning

confidence: 99%

“…It studies efficient similarity join on probabilistic sets based on several pruning techniques. Another recent work [39] proposes a simpler model for probabilistic set where the existence of elements are independent. However, to apply these models to the above-mentioned application scenarios, the user will face at least one of the following problems: (1) the probabilistic set needs exponentially large space to define if using the set-level model, (2) the similarity computation between two probabilistic sets requires time exponential in the size of the probabilistic sets, if using the element-level model, or (3) there are insufficient query processing techniques for pruning candidate sets especially when many sets in the database contains large numbers of elements.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Similarity query processing for probabilistic sets

Gao

Jin

Wang

et al. 2013

2013 IEEE 29th International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Abstract-Evaluating similarity between sets is a fundamental task in computer science. However, there are many applications in which elements in a set may be uncertain due to various reasons. Existing work on modeling such probabilistic sets and computing their similarities suffers from huge model sizes or significant similarity evaluation cost, and hence is only applicable to small probabilistic sets.In this paper, we propose a simple yet expressive model that supports many applications where one probabilistic set may have thousands of elements. We define two types of similarities between two probabilistic sets using the possible world semantics; they complement each other in capturing the similarity distributions in the cross product of possible worlds. We design efficient dynamic programming-based algorithms to calculate both types of similarities. Novel individual and batch pruning techniques based on upper bounding the similarity values are also proposed. To accommodate extremely large probabilistic sets, we also design sampling-based approximate query processing methods with strong probabilistic guarantees. We have conducted extensive experiments using both synthetic and real datasets, and demonstrated the effectiveness and efficiency of our proposed methods.

show abstract

“…Sets are ubiquitous in computer science and most importantly in the field of data management; they model among others transactions and scientific data, click streams and Web search data, text. Contemporary data management systems allow the definition of set-valued (or multi-valued) data attributes and support operations such as containment queries [1,23,37,38,42]. Joins are also extended to include predicates on sets (containment, similarity, equality, etc.)…”

Section: Introductionmentioning

confidence: 99%

Set containment join revisited

Bouros

Mamoulis

et al. 2015

Knowl Inf Syst

View full text Add to dashboard Cite

Given two collections of set objects R and S, the R ⊆ S set containment join returns all object pairs (r, s) ∈ R × S such that r ⊆ s. Besides being a basic operator in all modern data management systems with a wide range of applications, the join can be used to evaluate complex SQL queries based on relational division and as a module of data mining algorithms. The state-of-the-art algorithm for set containment joins (PRETTI) builds an inverted index on the right-hand collection S and a prefix tree on the left-hand collection R that groups set objects with common prefixes and thus, avoids redundant processing. In this paper, we present a framework which improves PRETTI in two directions. First, we limit the prefix tree construction by proposing an adaptive methodology based on a cost model; this way, we can greatly reduce the space and time cost of the join. Second, we partition the objects of each collection based on their first contained item, assuming that the set objects are internally sorted. We show that we can process the partitions and evaluate the join while building the prefix tree and the inverted index progressively. This allows us to significantly reduce not only the join cost, but also the maximum memory requirements during the join.

show abstract

Efficient processing of probabilistic set-containment queries on uncertain set-valued data

Cited by 5 publications

References 42 publications

Efficient processing of containment queries on nested sets

Efficient processing of containment queries on nested sets

Similarity query processing for probabilistic sets

Set containment join revisited

Contact Info

Product

Resources

About