Highly expressive declarative languages, such as datalog, are now commonly used to model the operational logic of dataintensive applications. The typical complexity of such datalog programs, and the large volume of data that they process, call for result explanation. Results may be explained through the tracking and presentation of data provenance, and here we focus on a detailed form of provenance (howprovenance), defining it as the set of derivation trees of a given fact. While informative, the size of such full provenance information is typically too large and complex (even when compactly represented) to allow displaying it to the user. To this end, we propose a novel top-k query language for querying datalog provenance, supporting selection criteria based on tree patterns and ranking based on the rules and database facts used in derivation. We propose an efficient novel algorithm based on (1) instrumenting the datalog program so that, upon evaluation, it generates only relevant provenance, and (2) efficient top-k (relevant) provenance generation, combined with bottom-up datalog evaluation. The algorithm computes in polynomial data complexity a compact representation of the top-k trees which may be explicitly constructed in linear time with respect to their size. We further experimentally study the algorithm performance, showing its scalability even for complex datalog programs where full provenance tracking is infeasible.

Data analytics often involves hypothetical reasoning: repeatedly modifying the data and observing the induced effect on the computation result of a datacentric application. Previous work has shown that fine-grained data provenance can help make such an analysis more efficient: instead of a costly re-execution of the underlying application, hypothetical scenarios are applied to a pre-computed provenance expression. However, storing provenance for complex queries and large-scale data leads to a significant overhead, which is often a barrier to the incorporation of provenance-based solutions. To this end, we present a framework that allows to reduce provenance size. Our approach is based on reducing the provenance granularity using user defined abstraction trees over the provenance variables; the granularity is based on the anticipated hypothetical scenarios. We formalize the tradeoff between provenance size and supported granularity of the hypothetical reasoning, and study the complexity of the resulting optimization problem, provide efficient algorithms for tractable cases and heuristics for others. We experimentally study the performance of our solution for various queries and abstraction trees. Our study shows that the algorithms generally lead to substantial speedup of hypothetical reasoning, with a reasonable loss of accuracy.

We consider in this paper static analysis of the possible executions of data-dependent applications, namely applications whose control flow is guided by a finite-state machine, as well as by the state of an underlying database. We note that previous work in this context has not addressed two important features of such analysis, namely analysis under hypothetical scenarios, such as changes to the application's state machine and/or to the underlying database, and the consideration of meta-data, such as cost or access privileges. Observing that semiring-based provenance has been proven highly effective in supporting these two features for database queries, we develop in this paper a semiring-based provenance framework for the analysis of data-dependent processes, accounting for hypothetical reasoning and meta-data. The development addresses two interacting new challenges: (1) combining provenance annotations for both information that resides in the database and information about external inputs (e.g., user choices) and (2) finitely capturing infinitely many process executions. We have implemented our framework as part of the PROPOLIS system.

No abstract

No abstract