Graph Processing Platforms at Scale: Practices and Experiences

Lim, Seung-Hwan; Lee, Sangkeun; Ganesh, Gautam; Brown, Tyler C.; Sukumar, Sreenivas R.

doi:10.1109/ispass.2015.7095783

Cited by 13 publications

(6 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Comprehensive and fair comparison of graph analysis systems involves with many challenges due to heterogeneity of systems including system designs, controllable parameters, the way of operating the system, and etc., as we discussed in our previous studies (Lim et al, 2015;Hong et al, 2015). Thus, in order to provide a general reference for relative performance difference between systems, we chose 6 real world datasets (Leskovec and Krevl, 2014) released by Stanford Network Analysis Project (SNAP) (Leskovec and Sosič, 2014), which are one of the most widely used data sets for graph analysis system performance benchmarking.…”

Section: Performance Evaluationsupporting

confidence: 68%

“…For both iterative and non-iterative graph mining algorithms, our results are very comparable to results from linearalgebra based implementations of the same algorithms. A comparison of our implementation with different linearalgebra based methods and map-reduce based methods for graph processing is presented in (Lim et al, 2015). The results in (Lim et al, 2015) along with the results in this paper clearly confirm that SPARQL-based implementation of graph mining algorithms can be unleashed on graph-data represented and hosted using Semantic Web standards and technologies.…”

Section: Performance Comparison Of Iterative Algorithmssupporting

confidence: 61%

“…Most researches conducted on graph analysis focused on achieving specific analytic operations (McColl et al, 2014;Lim et al, 2015;Shao et al, 2012); however, we envision that many applications in graph analysis require processing cyclic work-flows of disparate analytic operations. Realizing a holistic in-situ graph analysis system is a new challenge we face, and it has not been much explored so far.…”

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Enabling graph mining in RDF triplestores using SPARQL for holistic in-situ graph analysis

Lee

Sukumar

Hong

et al. 2016

Expert Systems with Applications

Self Cite

View full text Add to dashboard Cite

Graph analysis is now considered as a promising technique to discover useful knowledge in data with a new perspective. We envision that there are two dimensions of graph analysis: OnLine Graph Analytic Processing (OLGAP) and Graph Mining (GM) where each respectively focuses on subgraph pattern matching and automatic knowledge discovery in graph. As these two dimensions aim to complementarily solve complex problems, holistic in-situ graph analysis which covers both OLGAP and GM in a single system is critical for minimizing the burdens of operating multiple graph systems and transferring intermediate result-sets between those systems. Nevertheless, most existing graph analysis systems are only capable of one dimension of graph analysis. In this work, we take an approach to enabling GM capabilities (e.g., PageRank, connected-component analysis, node eccentricity, etc.) in RDF triplestores, which are originally developed to store RDF datasets and provide OLGAP capability. More specifically, to achieve our goal, we implemented six representative graph mining algorithms using SPARQL. The approach allows a wide range of available RDF data sets directly applicable for holistic graph analysis within a system. For validation of our approach, we evaluate performance of our implementations with nine real-world datasets and three different computing environments -a laptop computer, an Amazon EC2 instance, and a shared-memory Cray XMT2 URIKA-GD graph-processing appliance. The experimental results show that our implementation can provide promising and scalable performance for real world graph analysis in all tested environments. The developed software is publicly available in an open-source project that we initiated.

show abstract

Section: Performance Evaluationsupporting

confidence: 68%

Section: Performance Comparison Of Iterative Algorithmssupporting

confidence: 61%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Enabling graph mining in RDF triplestores using SPARQL for holistic in-situ graph analysis

Lee

Sukumar

Hong

et al. 2016

Expert Systems with Applications

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is very comparable to results from linear-algebra based implementations of the same algorithms. A comparison of our implementation with different linear-algebra based methods and map-reduce based methods for graph processing is presented in [24]. The results in [24] along with the results in this paper clearly confirm that SPARQL-based implementation of graph mining algorithms can be unleashed on graph-data represented and hosted using Semantic Web standards and technologies.…”

Section: Performance Evaluationsupporting

confidence: 60%

Graph mining meets the Semantic Web

Lee

Sukumar

Lim

2015

2015 31st IEEE International Conference on Data Engineering Workshops

Self Cite

View full text Add to dashboard Cite

The Resource Description Framework (RDF) and SPARQL Protocol and RDF Query Language (SPARQL) were introduced about a decade ago to enable flexible schema-free data interchange on the Semantic Web. Today, data scientists use the framework as a scalable graph representation for integrating, querying, exploring and analyzing data sets hosted at different sources. With increasing adoption, the need for graph mining capabilities for the Semantic Web has emerged. We address that need through implementation of three popular iterative Graph Mining algorithms (Triangle count, Connected component analysis, and PageRank). We implement these algorithms as SPARQL queries, wrapped within Python scripts. We evaluate the performance of our implementation on 6 real world data sets and show graph mining algorithms (that have a linear-algebra formulation) can indeed be unleashed on data represented as RDF graphs using the SPARQL query interface.

show abstract

“…As a result, RDF data can be represented as a labeled, directed multigraph, as shown in Figure 1. An RDF data graph can be presented by the equation below (Lim et al, 2015):…”

Section: Semantic Webmentioning

confidence: 99%

An Efficient Approach to Extract and Store Big Semantic Web Data Using Hadoop and Apache Spark GraphX

Salih Mohammed,

Ju Maa

2024

ADCAIJ

View full text Add to dashboard Cite

The volume of data is growing at an astonishingly high speed. Traditional techniques for storing and processing data, such as relational and centralized databases, have become inefficient and time-consuming. Linked data and the Semantic Web make internet data machine-readable. Because of the increasing volume of linked data and Semantic Web data, storing and working with them using traditional approaches is not enough, and this causes limited hardware resources. To solve this problem, storing datasets using distributed and clustered methods is essential. Hadoop can store datasets because it can use many hard disks for distributed data clustering; Apache Spark can be used for parallel data processing more efficiently than Hadoop MapReduce because Spark uses memory instead of the hard disk. Semantic Web data has been stored and processed in this paper using Apache Spark GraphX and the Hadoop Distributed File System (HDFS). Spark's in-memory processing and distributed computing enable efficient data analysis of massive datasets stored in HDFS. Spark GraphX allows graph-based semantic web data processing. The fundamental objective of this work is to provide a way for efficiently combining Semantic Web and big data technologies to utilize their combined strengths in data analysis and processing. First, the proposed approach uses the SPARQL query language to extract Semantic Web data from DBpedia datasets. DBpedia is a hugely available Semantic Web dataset built on Wikipedia. Secondly, the extracted Semantic Web data was converted to the GraphX data format; vertices and edges files were generated. The conversion process is implemented using Apache Spark GraphX. Third, both vertices and edge tables are stored in HDFS and are available for visualization and analysis operations. Furthermore, the proposed techniques improve the data storage efficiency by reducing the amount of storage space by half when converting from Semantic Web Data to a GraphX file, meaning the RDF size is around 133.8 and GraphX is 75.3. Adopting parallel data processing provided by Apache Spark in the proposed technique reduces the required data processing and analysis time. This article concludes that Apache Spark GraphX can enhance Semantic Web and Big Data technologies. We minimize data size and processing time by converting Semantic Web data to GraphX format, enabling efficient data management and seamless integration.

show abstract

Graph Processing Platforms at Scale: Practices and Experiences

Cited by 13 publications

References 38 publications

Enabling graph mining in RDF triplestores using SPARQL for holistic in-situ graph analysis

Enabling graph mining in RDF triplestores using SPARQL for holistic in-situ graph analysis

Graph mining meets the Semantic Web

An Efficient Approach to Extract and Store Big Semantic Web Data Using Hadoop and Apache Spark GraphX

Contact Info

Product

Resources

About