Graphalytics

Abstract

Graphs are increasingly used in industry, governance, and science. This has stimulated the appearance of many and diverse graph-processing platforms. Although platform diversity is beneficial, it also makes it very challenging to select the best platform for an application domain or one of its important applications, and to design new and tune existing platforms. Continuing a long tradition of using benchmarking to address such challenges, in this work we present our vision for Graphalytics, a big data benchmark for graphprocessing platforms. We have already benchmarked with Graphalytics a variety of popular platforms, such as Giraph, GraphX, and Neo4j.

Fregel: a functional domain-specific language for vertex-centric large-scale graph processing

“…Many vertex-centric graph processing frameworks have been proposed, including Giraph, 3 GraphLab (Low et al 2012), GPS (Salihoglu & Widom 2013), GraphX (Gonzalez et al 2014), and Pregel+ (Yan et al 2014b). Many other frameworks can be found from extensible surveys on large graph processing (Khan & Elnikety 2014; McCune et al , 2015; Yan et al , 2016; Khan, 2017; Yan et al , 2017; Kalavri et al , 2018; Liu & Khan, 2018) and experimental evaluations of these frameworks (Han et al 2014; Lu et al , 2014; Guo et al , 2014; Satish et al , 2014; Capota et al , 2015; Gao et al , 2015; Verma et al , 2017).…”

Big data analytics on Apache Spark

“…13 Also, IBM SystemML 14 was open sourced and there is a plan to collaborate with Databricks to enhance machine learning capabilities in Spark's MLlib. Furthermore, Microsoft announced a major commitment 15 to support Apache Spark through Microsoft's platforms and services such as Azure HDInsight 16 and Microsoft R Server. 17 There are considerable case studies of using Apache Spark for different kinds of applications: e.g., planning and optimization of video advertising campaigns at Eyeview [18], categorizing and prioritizing social media interactions in real time at Toyota Motor Sales, USA [50], predicting the offlining of digital media at NBC Universal [14] and real-time anomaly detection at ING banking [15].…”

“…In addition to the research highlights we presented in the previous sections, there are other research works which have been done using Apache Spark as a core engine for solving data problems in machine learning and data mining [5,36], graph processing [16], genomic analysis [60,65], time series data [71], smart grid data [73], spatial data processing [87], scientific computations of satellite data [67], large-scale biological sequence alignment [97] and data discretization [68]. There are also some recent works on using Apache Spark for deep learning [46,64].…”