Distributed Graph Analytics with Datalog Queries in Flink

Fast datalog evaluation for batch and stream graph processing

“…Listing 10 shows the concrete implementation of Algorithm 3. The implementation is somewhat similar to the one we discussed in [21] except for a few differences: a) the groupBy key(s) is the grouping key of the aggregation (instead of using all the fields), and b) coGroup with AggrMinusCoGroupfunction performs the operation instead of a normal minus.…”

“…2. Single-Source Shortest Path (SSSP): Figures 17,18,19,and 20 show the result of our experimental comparison using the SSSP program with RMAT datasets of varying sizes and configurations (as provided in Section 7.1). For Nexus, the increase in the execution time is within 1.1 − 1.8× for each twofold increase in the dataset size for configuration 1; 1.0 − 1.7× for configuration 2; 1.1 − 1.8× for configuration 3; and 1.0 − 1.6× for configuration 4.…”

“…Recently, we proposed Cog [20] to tackle this problem. Cog is a Datalog's recursive program evaluation system that implements a semi-naive algorithm [3] in a cyclic dataflow system (namely Apache Flink [10]) using its incremental iterations.…”

“…In addition, Nexus provides debugging capabilities to understand recursive aggregate programs as well as to explain the outcomes of their derivations. Nexus is an extension of Cog [20]. It comes with a number of new and novel features.…”

“…-We exhaustively benchmark Nexus using five common graph analytics tasks (including two programs that contain aggregation in recursion) with synthetic and real-world datasets. We compare Nexus to acyclic dataflow-based BigDatalog [33] and to Cog [20]. The results show that Nexus is up to 5.5× faster than the baseline BigDatalog and 3× faster than Cog.…”