Representing and Querying Correlated Tuples in Probabilistic Databases

Abstract

Probabilistic databases have received considerable attention recently due to the need for storing uncertain data produced by many real world applications. The widespread use of probabilistic databases is hampered by two limitations: (1) current probabilistic databases make simplistic assumptions about the data (e.g., complete independence among tuples) that make it difficult to use them in applications that naturally produce correlated data, and (2) most probabilistic databases can only answer a restricted subset of the queries that can be expressed using traditional query languages. We address both these limitations by proposing a framework that can represent not only probabilistic tuples, but also correlations that may be present among them. Our proposed framework naturally lends itself to the possible world semantics thus preserving the precise query semantics extant in current probabilistic databases. We develop an efficient strategy for query evaluation over such probabilistic databases by casting the query processing problem as an inference problem in an appropriately constructed probabilistic graphical model. We present several optimizations specific to probabilistic databases that enable efficient query evaluation. We validate our approach by presenting an experimental evaluation that illustrates the effectiveness of our techniques at answering various queries using real and synthetic datasets.

Database Support for Probabilistic Attributes and Tuples

“…A similar concept is used in many tuple uncertainty models to track correlations between tuples. [9] uses lineage and [14] uses factor tables to capture such dependencies. As we are interested in capturing historical dependencies between attributes of tuples, our concept of dependencies is different from this related work, which capture these dependencies on a per tuple basis.…”

“…The ProbView system [16] took a similar approach by propagating the formulas necessary to evaluating the resulting probabilities. Sen et al have more recently proposed an alternative approach to represent tuple correlations using probabilistic graphical models [14]. They use factored representations of the relations to represent their dependencies.…”

Threshold-based probabilistic top-k dominating queries

“…However, because of the independence assumption, it is non-trivial to extend our exact algorithm to tackle the problem against dataset with correlations. As a possible future work, we will consider to develop efficient exact algorithm based on the graph model [38,15] which can effectively capture the correlations of the uncertain dataset.…”

“…Uncertainty is inherent in such applications due to various factors such as data randomness and incompleteness, limitation of equipment, and delay or loss in data transfer. A number of issues have been recently addressed; these include modeling uncertainty [2,36], query evaluation [10,13,14,37], indexing [11,41], top-k queries [22,35,39,42], skyline queries [34], joins [26,27], nearest neighbor query [5,9,27], clustering [28,30], etc.…”

Probabilistic Threshold Range Aggregate Query Processing over Uncertain Data

“…General issues in modelling and managing uncertain data are addressed by Dey and Sarkar in [4], Lee in [11], and Antova, Koch, and Olteanu in [6]. Querying uncertain data by the probabilistic paradigm has been investigated by Dalvi and Suciu in [2] and Sen and Deshpande in [17]. Very recently Dalvi and Suciu [3] have shown that the problem of query evaluation over probabilistic databases is either P T IM E or #P -complete.…”