Towards scalable RDF graph analytics on MapReduce

Abstract

In order to exploit the growing amount of RDF data in decisionmaking, there is an increasing demand for analytics-style processing of such data. RDF data is modeled as a labeled graph that represents a collection of binary relations (triples). In this context, analytical queries can be interpreted as consisting of three main constructs namely pattern matching, grouping and aggregation, and require several join operations to reassemble them into n-ary relations relevant to the given query, unlike traditional OLAP systems where data is suitably organized. MapReduce-based parallel processing systems like Pig have gained success in processing scalable analytical workloads. However, these systems offer only relational algebra style operators which would require an iterative n-tuple reassembly process in which intermediate results need to be materialized. This leads to high I/O costs that negatively impacts performance. In this paper, we propose UDFs that (i) re-factor analytical processing on RDF graphs in a way that enables more parallelized processing (ii) perform a look-ahead processing to reduce the cost of subsequent operators in the query execution plan. These functions have been integrated into the Pig Latin function library and the experimental results show up to 50% improvement in execution times for certain classes of queries. An important impact of this work is that it could serve as the foundation for additional physical operators in systems such as Pig for more efficient graph processing.

An Intermediate Algebra for Optimizing RDF Graph Pattern Matching on MapReduce

“…Our previous work, RAPID [13] focused on optimizing analytical processing of RDF data on Pig. RAPID+ [10] extended Pig with UDFs to enable TripleGroup-based processing. In this work, we provide formal semantics to integrate TripleGroups as first-class citizens, and present operators for graph pattern matching.…”

“…In our previous work [10], we proposed an approach to exploit star sub patterns by re-interpreting star-joins using a grouping-based join algorithm. It can be observed that performing a group by Subject yields groups of tuples or TripleGroups that represent all the star sub graphs in the database.…”

Robust Runtime Optimization and Skew-Resistant Execution of Analytical SPARQL Queries on Pig

“…PIGSparQL [19] performs a direct mapping of SPARQL to Pig without focusing on optimization. RAPID+ [17] provides a limited form of on-the-fly optimization where look-ahead processing tries to combine multiple subsequent join steps. The adaptiveness of this approach is however limited compared to our sampling based run-time query optimization.…”

Configurable and Extensible Multi-flows for Providing Analytics as a Service on the Cloud

“…Forecasts [14] suggest that cloud computing would grow at a very fast rate over the next few years. Cloud computing has been used for a variety of tasks such as parallel data mining [15] and graph analytics [16].…”