Hypothetical Reasoning via Provenance Abstraction

Abstract

Data analytics often involves hypothetical reasoning: repeatedly modifying the data and observing the induced effect on the computation result of a datacentric application. Previous work has shown that fine-grained data provenance can help make such an analysis more efficient: instead of a costly re-execution of the underlying application, hypothetical scenarios are applied to a pre-computed provenance expression. However, storing provenance for complex queries and large-scale data leads to a significant overhead, which is often a barrier to the incorporation of provenance-based solutions. To this end, we present a framework that allows to reduce provenance size. Our approach is based on reducing the provenance granularity using user defined abstraction trees over the provenance variables; the granularity is based on the anticipated hypothetical scenarios. We formalize the tradeoff between provenance size and supported granularity of the hypothetical reasoning, and study the complexity of the resulting optimization problem, provide efficient algorithms for tractable cases and heuristics for others. We experimentally study the performance of our solution for various queries and abstraction trees. Our study shows that the algorithms generally lead to substantial speedup of hypothetical reasoning, with a reasonable loss of accuracy.

COBRA: Compression Via Abstraction of Provenance for Hypothetical Reasoning

“…Optimization Problem The problem we have studied in [4] is as follows: Given a provenance polynomial and abstraction tree over (subsets of) its variables, find a choice of abstraction that reduces the provenance size, while maximizing the expressiveness of the abstraction; we next explain both measures. First, the provenance size is measured by the number of monomials in the resulting provenance polynomial.…”

“…In this case the optimization problem is solvable in polynomial time complexity. In a nutshell, the algorithm traverses the abstraction tree in a bottom-up fashion, and using dynamic programming, computes an abstraction for the sub-tree rooted by each one of the inner nodes (see [4] for full details).…”

“…In a paper that is to appear in SIGMOD '19 [4], we presented a framework for the reduction of provenance size for hypothetical reasoning. The framework is based on the notion of abstraction; the main idea is that instead of assigning a distinct variable per cell/tuple, we can often group variables together, forming an abstract "meta-variable".…”

“…By doing so, we decrease the degree of freedom for hypotheticals (because now we are forced to assign a single value to all grouped variables), but we also gain in provenance size: distinct monomials may become identical, in which case they are compactly represented by a single monomial (by summing their coefficients). Whether or not it makes sense to group variables together depends on their semantics; to enable meaningful abstraction we introduced in [4] the notion of abstraction trees, which resemble ontologies over the provenance variables to guide and restrict the allowed groupings.…”

“…The system allows examining the effect of the provenance compression on the anticipated analysis results. The framework is based on the algorithm presented in [4], and designed to assist the meta-analysis determine the desired bound over the compressed provenance size and the construction of the abstraction tree. This is done by presenting the changes in the analysis query results using valuation of the compressed provenance with respect to valuation of the full provenance.…”