Precise complexity analysis for efficient datalog queries

Abstract

Given a set of Datalog rules, facts, and a query, answers to the query can be inferred bottom-up starting with the facts or top-down starting with the query. For efficiently answering the query, topdown evaluation is extended with tabling that stores the results of the subqueries encountered, and bottom-up evaluation is done on rules transformed based on demand from the query.This paper describes precise time and space complexity analysis for efficiently answering Datalog queries, and precise relationships between top-down evaluation with tabling and bottom-up evaluation driven by demand. We first present a systematic method for precisely calculating the worst-case time and space complexities of top-down evaluation with tabling. We then describe a method for transforming the rules for efficiently answering queries using bottom-up evaluation of the transformed rules; the method is akin to the magic set transformation, but is simpler and produces simpler rules that yield exponentially smaller space in the number of arguments of predicates. Next, we establish precise relationships between top-down evaluation with tabling and bottom-up evaluation of rules transformed based on demand. Finally, we support our analyses and comparisons through experiments on benchmarks from OpenRuleBench.

More efficient datalog queries

“…We show that top-down evaluation using subsumptive tabling is equal to or better than using variant tabling in both time and space complexities. Using this result and the relationship between top-down evaluation with variant tabling and bottom-up evaluation after MST as shown in [27], we show that subsumptive tabling is equal to or better than MST in both time and space complexities. Then, we characterize a class of Datalog rules for which subsumptive tabling is guaranteed to be better than variant tabling in both time and space complexities.…”

“…Demand transformation [27] is similar to MST, with better space complexity in program size, but the same time complexity. There exists no transformation such that the bottom-up evaluation of transformed rules achieves the performance of subsumptive tabling.…”

“…We give complexity analyses for top-down evaluation with subsumptive tabling by determining the binding patterns of arguments for queries, and the subqueries that are guaranteed to reuse answers from subsuming subqueries, and then extending the analysis for variant tabling [27] to subsumptive tabling. We show that top-down evaluation using subsumptive tabling is equal to or better than using variant tabling in both time and space complexities.…”

Deriving predicate statistics in datalog

“…[19] studied the precise time and space complexity for efficient query answering in Datalog. Given a Datalog program and a query, complexity formulas for different evaluations strategies (e.g., top-down evaluation with tabling or bottom-up evaluation with demand driven transformation) are computed and compared.…”

“…Given a Datalog program and a query, complexity formulas for different evaluations strategies (e.g., top-down evaluation with tabling or bottom-up evaluation with demand driven transformation) are computed and compared. This approach is complementary to ours in the sense that SDP can supply parameter estimates, such as inand out-degrees of predicates, which are needed for the complexity analysis of [19] to be practically applicable.…”