Advances in Large-Scale RDF Data Management

Abstract

Abstract.One of the prime goals of the LOD2 project is improving the performance and scalability of RDF storage solutions so that the increasing amount of Linked Open Data (LOD) can be efficiently managed. Virtuoso has been chosen as the basic RDF store for the LOD2 project, and during the project it has been significantly improved by incorporating advanced relational database techniques from MonetDB and Vectorwise, turning it into a compressed column store with vectored execution. This has reduced the performance gap ("RDF tax") between Virtuoso's SQL and SPARQL query performance in a way that still respects the "schema-last" nature of RDF. However, by lacking schema information, RDF database systems such as Virtuoso still cannot use advanced relational storage optimizations such as table partitioning or clustered indexes and have to execute SPARQL queries with many selfjoins to a triple table, which leads to more join effort than needed in SQL systems. In this chapter, we first discuss the new column store techniques applied to Virtuoso, the enhancements in its cluster parallel version, and show its performance using the popular BSBM benchmark at the unsurpassed scale of 150 billion triples. We finally describe ongoing work in deriving an "emergent" relational schema from RDF data, which can help to close the performance gap between relational-based and RDF-based storage solutions. General ObjectivesOne of the objectives of the LOD2 EU project is to boost the performance and the scalability of RDF storage solutions so that it can, efficiently manage huge datasets of Linked Open Data (LOD). However, it has been noted that given similar data management tasks, relational database technology significantly outperformed RDF data stores. One controlled scenario in which the two technologies can be compared is the BSBM benchmark [2], which exists equivalent relational and RDF variants. As illustrated in Fig. 1, while the SQL systems can process by up to 40-175K QMpH, the Triple stores can only reach 1-10K QMpH, showing a factor of 15-40 of performances difference.

HDTQ: Managing RDF Datasets in Compressed Space

“…Regardless of the underneath model (based on a relational schema, implementing a native index or a NOSQL solution), RDF stores often speed up quad-based queries by indexing different combinations of the subject, predicate, object and graph elements in RDF [13]. Virtuoso [5] implements quads in a column-based relational store, with two full indexes over the RDF quads, with PSOG and POSG order, and 3 projections SP, OP and GS. The well-known Apache Jena TDB 5 stores RDF datasets using 6 B+Trees indexes, namely SPOG, POSG, OSPG, GSPO, GPOS and GOSP.…”

Everything you always wanted to know about compiled and vectorized queries but were afraid to ask

“…If the query optimizer's decision about whether to break up a pipeline is correct (which is non-trivial [24]), Peloton can be faster than both standard models. [27] push interpretation 2001 MonetDB [9] n/a vectorization 1996 VectorWise [7] pull vectorization 2005 Virtuoso [8] push vectorization 2013 Hique [18] n/a compilation 2010 HyPer [28] push compilation 2011 Hekaton [12] pull compilation 2014…”

“…One advantage of the push model is that it enables DAG-structured query plans (as opposed to trees), i.e., an operator may push its output to more than one consumer [27]. Push-based execution also has advantages in distributed query processing with Exchange operators, which is one of the reasons it has been adopted by Virtuoso [8]. One downside of the push model is that it is slightly less flexible in terms of control flow: A merge-sort, for example, has to fully materialize one input relation.…”

Linked Data for Language-Learning Applications

“…The technology for RDF triple stores is not as mature as for relational databases and this is reflected in their performance as witnessed by the so-called "RDF tax", although recent work has been done to improve this (Boncz et al, 2014). Performance for this prototype was also affected by the quality of the data contained in the database and the type of query performed.…”