Database Queries that Explain their Work

Abstract

Provenance for database queries or scientific workflows is often motivated as providing explanation, increasing understanding of the underlying data sources and processes used to compute the query, and reproducibility, the capability to recompute the results on different inputs, possibly specialized to a part of the output. Many provenance systems claim to provide such capabilities; however, most lack formal definitions or guarantees of these properties, while others provide formal guarantees only for relatively limited classes of changes. Building on recent work on provenance traces and slicing for functional programming languages, we introduce a detailed tracing model of provenance for multiset-valued Nested Relational Calculus, define trace slicing algorithms that extract subtraces needed to explain or recompute specific parts of the output, and define query slicing and differencing techniques that support explanation. We state and prove correctness properties for these techniques and present a proof-of-concept implementation in Haskell.Comment: PPDP 201

Hypothetical Reasoning via Provenance Abstraction

“…There are multiple other lines of work aiming at reducing provenance size. One such approach is to track only parts of the provenance that are of interest, e.g., based on user specification [33,31,21,19]. Such selective provenance tracking is complementary to ours, as we start from a polynomial which may reflect full or partial provenance, invariably for our approach.…”

Language-integrated provenance

“…A trace resembles the shape of the query but also records important datadependent decisions that were made -or rather will be made -during query execution, such as whether a filter condition held. The trace datatype is inspired by previous work on program and query slicing [Cheney et al, 2014a;Perera et al, 2012] with two major differences: The traces of lists and records are lists and records of traces; in other words, trace information is accumulated in the leaves of a tree of list and record constructors. This goes hand in hand with not representing variable binding explicitly in the trace.…”

“…Provenance is also related to query and program slicing [Cheney, 2007;Cheney et al, 2011]. We discuss this particular relationship in more detail in Chapter 6, which implements data provenance on top of tracing and was inspired by work on program and query slicing [Cheney et al, 2014a;Ricciotti et al, 2017]. The sort of provenance supported by the languages described in this dissertation may help in debugging queries but the focus is data.…”

Language-integrated provenance

“…We intend to explore this further (and consider alternative models). Cheney et al [2014a] presented a general form of provenance for nested relational calculus based on execution traces, and showed how such traces can be used to provide "slices" that explain specific results. While this model appears to generalize all of the aforementioned approaches, it appears nontrivial to implement by translation to relational queries, because it is not obvious how to represent the traces in this approach in a relational data model.…”