Dissociation and propagation for approximate lifted inference with standard relational database management systems

Abstract

Probabilistic inference over large data sets is a challenging data management problem since exact inference is generally #P-hard and is most often solved approximately with sampling-based methods today. This paper proposes an alternative approach for approximate evaluation of conjunctive queries with standard relational databases: In our approach, every query is evaluated entirely in the database engine by evaluating a fixed number of query plans, each providing an upper bound on the true probability, then taking their minimum. We provide an algorithm that takes into account important schema information to enumerate only the minimal necessary plans among all possible plans. Importantly, this algorithm is a strict generalization of all known PTIME self-join-free conjunctive queries: A query is in PTIME if and only if our algorithm returns one single plan. Furthermore, our approach is a generalization of a family of efficient ranking methods from graphs to hypergraphs. We also adapt three relational query optimization techniques to evaluate all necessary plans very fast. We give a detailed experimental evaluation of our approach and, in the process, provide a new way of thinking about the value of probabilistic methods over non-probabilistic methods for ranking query answers. We also note that the techniques developed in this paper apply immediately to lifted inference from statistical relational models since lifted inference corresponds to PTIME plans in probabilistic databases.

Anytime Approximation in Probabilistic Databases via Scaled Dissociations

“…We next introduce our basic notions related to probabilistic query evaluation, lineages, dissociations, as well as the influence of variables on those lineages. We loosely follow the notation and concepts of [46] but also include more recent concepts from [14,16,17,34], [19][20][21] and [27,39].…”

“…Recent work [19] suggests socalled "dissociation"-based bounds for monotone Boolean formulas. These bounds were shown to generalize modelbased bounds, but have so far only been applied as one-shot approximations at the query level [19][20][21]. In addition, empirical results of [19][20][21] showed that, while the upper bounds appear to work well (they are provably better than all modelbased bounds), the lower "symmetric" bounds obtained by plan-level dissociations usually are poor approximations.…”

“…Alternatively, one may also generate a symmetric dissociation (SD) LB [20,21] in which all occurrences get the same share, i.e., p…”

“…For the resulting upper bounds, we are already guaranteed to improve over the model-based version. These are proven to be the optimal "oblivious" choice in [21], and hence they do not require further optimization. By contrast, the constraint set for the lower bounds results in an entire continuum of possible assignments (see Example 2), some of which correspond to model-based bounds.…”

Query-driven on-the-fly knowledge base construction

“…Query-time IE, as pursued in our work, resembles the notion of querytime inference over probabilistic databases [18,53]. These methods operate on uncertain relational data as well as uncertain rules, and support flexible forms of top-k queries [17] and general inference [23,24,31]. However, all of these approaches require a relational schema that underlies the KB.…”