Fast datalog evaluation for batch and stream graph processing

Imran, Muhammad; Gévay, Gábor; Quiané-Ruiz, Jorge-Arnulfo; Markl, Volker

doi:10.1007/s11280-021-00960-w

Cited by 9 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The de-facto datalog evaluation method, which virtually all recent reasoners [5,10,11,13,15,17] abide by, is to rewrite datalog rules into relational algebra, a well-known technique, to efficiently compute their evaluation due to the extensive industrial and academic research poured into developing data processing frameworks that handle very large amounts of data, and the techniques that have arisen from those.…”

Section: Relational Algebra Rewriting Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A Differential Datalog Interpreter

Stephenson

2023

Software

View full text Add to dashboard Cite

The core reasoning task for datalog engines is materialization, the evaluation of a datalog program over a database alongside its physical incorporation into the database itself. The de-facto method of computing is through the recursive application of inference rules. Due to it being a costly operation, it is a must for datalog engines to provide incremental materialization; that is, to adjust the computation to new data instead of restarting from scratch. One of the major caveats is that deleting data is notoriously more involved than adding since one has to take into account all possible data that has been entailed from what is being deleted. Differential dataflow is a computational model that provides efficient incremental maintenance, notoriously with equal performance between additions and deletions, and work distribution of iterative dataflows. In this paper, we investigate the performance of materialization with three reference datalog implementations, out of which one is built on top of a lightweight relational engine, and the two others are differential-dataflow and non-differential versions of the same rewrite algorithm with the same optimizations. Experimental results suggest that monotonic aggregation is more powerful than ascenting merely the powerset lattice.

show abstract

Section: Relational Algebra Rewriting Methodsmentioning

confidence: 99%

“…Flink, unlike Spark, supports iteration, so implementing reasoning did not need to extend the core of the underlying framework. The most successful attempt at creating a distributed implementation has been Nexus [17], which is also built on Flink and makes use of its most advanced feature, incremental stream processing.…”

Section: Related Workmentioning

confidence: 99%

A Differential Datalog Interpreter

Stephenson

2023

Software

View full text Add to dashboard Cite

show abstract

“…The de-facto datalog evaluation method, that virtually all recent reasoners [19,20,27,28,30,31] abide by, is to rewrite datalog rules into relational algebra, a well-known technique, to efficiently compute their evaluation, due to the extensive industrial and academic research poured into developing data processing frameworks that handle very large amounts of data, and the techniques that have arisen from those.…”

Section: Relational Algebra Rewriting Methodsmentioning

confidence: 99%

“…Flink, unlike Spark, supports iteration, so implementing reasoning did not need to extend the core of the underlying framework. The most successful attempt at creating a distributed implemention has been Nexus [19], that is also built on Flink, and makes use of its most advanced feature, incremental stream processing.…”

Section: Related Workmentioning

confidence: 99%

A Differential Datalog Interpreter

Stephenson

2023

Preprint

View full text Add to dashboard Cite

The core reasoning task for datalog engines is materialization, the evaluation of a datalog program over a database alongside its physical incorporation into the database itself. The de-facto method of computing it, is through the recursive application of inference rules. Due to it being a costly operation, it is a must for datalog engines to provide incremental materialization, that is, to adjust the computation to new data, instead of restarting from scratch. One of the major caveats, is that deleting data is notoriously more involved than adding, since one has to take into account all possible data that has been entailed from what is being deleted. Differential Dataflow is a computational model that provides efficient incremental maintenance, notoriously with equal performance between additions and deletions, and work distribution, of iterative dataflows. In this paper we investigate the performance of materialization with three reference datalog implementations, out of which one is built on top of a lightweight relational engine, and the two others are differential-dataflow and non-differential versions of the same rewrite algorithm, with the same optimizations.

show abstract

“…Another relevant direction is incremental graph pattern matching [3,6,12,21,25,26,28,29], where the goal is to find and maintain patterns in an updating graph.…”

Section: Related Workmentioning

confidence: 99%

Temporal graph patterns by timed automata

Aghasadeghi¹,

Bussche²,

Stoyanovich³

2022

Preprint

View full text Add to dashboard Cite

Temporal graphs represent graph evolution over time, and have been receiving considerable research attention. Work on expressing temporal graph patterns or discovering temporal motifs typically assumes relatively simple temporal constraints, such as journeys or, more generally, existential constraints, possibly with finite delays. In this paper we propose to use timed automata to express temporal constraints, leading to a general and powerful notion of temporal basic graph pattern (BGP).The new difficulty is the evaluation of the temporal constraint on a large set of matchings. An important benefit of timed automata is that they support an iterative state assignment, which can be useful for early detection of matches and pruning of non-matches. We introduce algorithms to retrieve all instances of a temporal BGP match in a graph, and present results of an extensive experimental evaluation, demonstrating interesting performance trade-offs. We show that an on-demand algorithm that processes total matchings incrementally over time is preferable when dealing with cyclic patterns on sparse graphs. On acyclic patterns or dense graphs, and when connectivity of partial matchings can be guaranteed, the best performance is achieved by maintaining partial matchings over time and allowing automaton evaluation to be fully incremental. The code and datasets are available at http: //github.com/amirpouya/TABGP.

show abstract

Fast datalog evaluation for batch and stream graph processing

Cited by 9 publications

References 31 publications

A Differential Datalog Interpreter

A Differential Datalog Interpreter

A Differential Datalog Interpreter

Temporal graph patterns by timed automata

Contact Info

Product

Resources

About