DBpedia SPARQL Benchmark – Performance Assessment with Real Queries on Real Data

Abstract

Abstract. Triple stores are the backbone of increasingly many Data Web applications. It is thus evident that the performance of those stores is mission critical for individual projects as well as for data integration on the Data Web in general. Consequently, it is of central importance during the implementation of any of these applications to have a clear picture of the weaknesses and strengths of current triple store implementations. In this paper, we propose a generic SPARQL benchmark creation procedure, which we apply to the DBpedia knowledge base. Previous approaches often compared relational and triple stores and, thus, settled on measuring performance against a relational database which had been converted to RDF by using SQL-like queries. In contrast to those approaches, our benchmark is based on queries that were actually issued by humans and applications against existing RDF data not resembling a relational schema. Our generic procedure for benchmark creation is based on query-log mining, clustering and SPARQL feature analysis. We argue that a pure SPARQL benchmark is more useful to compare existing triple stores and provide results for the popular triple store implementations Virtuoso, Sesame, Jena-TDB, and BigOWLIM. The subsequent comparison of our results with other benchmark results indicates that the performance of triple stores is by far less homogeneous than suggested by previous benchmarks.

User interfaces for semantic authoring of textual content: A systematic literature review

“…The W3C's Web Content Accessibility Guidelines (WCAG) 44 explain how to make Web content more accessible to people with disabilities. As part of WCAG, Authoring Tool Accessibility Guidelines (ATAG) 45 and more specifically Accessible Rich Internet Applications (WAI-ARIA) 46 suite, define how authoring tools should support accessibility requirements. Consequently, a challenge is to apply and extend the series of accessibility guidelines proposed in ATAG for the purpose of designing accessible SCA UIs.…”

“…On the backend side, a variety of triple stores were developed and their performance and maturity improves steadily (cf. recent triple store benchmarking efforts such as the DBpedia benchmark [46]). Similarly tools and algorithms for linking data and schemata are progressing and approaches are deployed for the use on the emerging Web of Data [48].…”

ParlBench: A SPARQL Benchmark for Electronic Publishing Applications

“…Both the Berlin and SP 2 Bench use synthetically generated data sets, whereas, the DBpedia SPARQL Benchmark (DBPSB) [2] uses a real data set, DBpedia [8]. In addition to using a real data set, the DBPSB benchmark uses real queries that were issued by humans and applications against DBpedia.…”

“…The existing application benchmarks for RDF store systems often employ techniques developed by the Transaction Processing Performance Council [4] (TPC) for relational databases and use synthetically generated data sets for their workloads. However, performance characteristics for loading and querying such data may differ from those that were measured on real life data sets, as it was shown by the DBpedia benchmark [2] on DBpedia [8]. To the best of our knowledge, among the existing benchmarks for RDF store systems, only the DBpedia benchmark provides a real data set.…”

Sharing Statistics for SPARQL Federation Optimization, with Emphasis on Benchmark Quality

“…Most recently, [6] addressed some of these problems by using real data and real queries from DBpedia. [7] has developed a benchmark for the federated case.…”