Anytime Approximation in Probabilistic Databases via Scaled Dissociations

Heuvel, Maarten Van den; Ivanov, Peter; Gatterbauer, Wolfgang; Geerts, Floris; Theobald, Martin

doi:10.1145/3299869.3319900

Cited by 3 publications

(1 citation statement)

References 41 publications

(121 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While some top-k approaches support this functionality or can easily be extended to do so, others rely on knowing k for pruning lower-ranked results. In order to more clearly distinguish between them, we will refer to ranked-enumeration algorithms also as "any-k" join algorithms as a shorthand for "anytime top-k. " Despite being reminiscent of the general concept of an anytime algorithm [15,22,32,96], any-k algorithms are not approximating the query result [69]. Instead, they reside squarely at the intersection of top-k and optimal joins, and we will discuss how they are impacted by ideas from both.…”

Section: Part 3: Ranked Enumeration Over Joins ("Any-k")mentioning

confidence: 99%

Optimal Join Algorithms Meet Top-k

Tziavelis

Gatterbauer

Riedewald

2020

Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

Top-k queries have been studied intensively in the database community and they are an important means to reduce query cost when only the "best" or "most interesting" results are needed instead of the full output. While some optimality results exist, e.g., the famous Threshold Algorithm, they hold only in a fairly limited model of computation that does not account for the cost incurred by large intermediate results and hence is not aligned with typical database-optimizer cost models. On the other hand, the idea of avoiding large intermediate results is arguably the main goal of recent work on optimal join algorithms, which uses the standard RAM model of computation to determine algorithm complexity. This research has created a lot of excitement due to its promise of reducing the time complexity of join queries with cycles, but it has mostly focused on full-output computation. We argue that the two areas can and should be studied from a unified point of view in order to achieve optimality in the common model of computation for a very general class of top-k-style join queries. This tutorial has two main objectives. First, we will explore and contrast the main assumptions, concepts, and algorithmic achievements of the two research areas. Second, we will cover recent, as well as some older, approaches that emerged at the intersection to support efficient ranked enumeration of join-query results. These are related to classic work on k-shortest path algorithms and more general optimization problems, some of which dates back to the 1950s. We demonstrate that this line of research warrants renewed attention in the challenging context of ranked enumeration for general join queries.

show abstract

Section: Part 3: Ranked Enumeration Over Joins ("Any-k")mentioning

confidence: 99%

Optimal Join Algorithms Meet Top-k

Tziavelis

Gatterbauer

Riedewald

2020

Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data

Self Cite

View full text Add to dashboard Cite

show abstract

Probabilistic Reasoning at Scale: Trigger Graphs to the Rescue

Tsamoura

Lee

Urbani

2023

Proc. ACM Manag. Data

View full text Add to dashboard Cite

The role of uncertainty in data management has become more prominent than ever before, especially because of the growing importance of machine learning-driven applications that produce large uncertain databases. A well-known approach to querying such databases is to blend rule-based reasoning with uncertainty. However, techniques proposed so far struggle with large databases. In this paper, we address this problem by presenting a new technique for probabilistic reasoning that exploits Trigger Graphs (TGs) -- a notion recently introduced for the non-probabilistic setting. The intuition is that TGs can effectively store a probabilistic model by avoiding an explicit materialization of the lineage and by grouping together similar derivations of the same fact. Firstly, we show how TGs can be adapted to support the possible world semantics. Then, we describe techniques for efficiently computing a probabilistic model and formally establish the correctness of our approach. We also present an extensive empirical evaluation using a prototype called LTGs. Our comparison against other leading engines shows that LTGs is not only faster, even against approximate reasoning techniques, but can also reason over probabilistic databases that existing engines cannot scale to.

show abstract

Minimally Factorizing the Provenance of Self-join Free Conjunctive Queries

Makhija,

Gatterbauer

2024

Proc. ACM Manag. Data

View full text Add to dashboard Cite

We consider the problem of finding the minimal-size factorization of the provenance of self-join-free conjunctive queries, i.e.,we want to find a formula that minimizes the number of variable repetitions. This problem is equivalent to solving the fundamental Boolean formula factorization problem for the restricted setting of the provenance formulas of self-join free queries. While general Boolean formula minimization is Σ p 2 -complete, we show that the problem is NP-Complete in our case. Additionally, we identify a large class of queries that can be solved in PTIME, expanding beyond the previously known tractable cases of read-once formulas and hierarchical queries. We describe connections between factorizations, Variable Elimination Orders (VEOs), and minimal query plans. We leverage these insights to create an Integer Linear Program (ILP) that can solve the minimal factorization problem exactly. We also propose a Max-Flow Min-Cut (MFMC) based algorithm that gives an efficient approximate solution. Importantly, we show that both the Linear Programming (LP) relaxation of our ILP, and our MFMC-based algorithm are always correct for all currently known PTIME cases. Thus, we present two unified algorithms (ILP and MFMC) that can both recover all known PTIME cases in PTIME, yet also solve NP-Complete cases either exactly (ILP) or approximately (MFMC), as desired.

show abstract

Anytime Approximation in Probabilistic Databases via Scaled Dissociations

Cited by 3 publications

References 41 publications

Optimal Join Algorithms Meet Top-k

Optimal Join Algorithms Meet Top-k

Probabilistic Reasoning at Scale: Trigger Graphs to the Rescue

Minimally Factorizing the Provenance of Self-join Free Conjunctive Queries

Contact Info

Product

Resources

About