WebPIE: A Web-scale Parallel Inference Engine using MapReduce

Urbani, Jacopo; Kotoulas, Spyros; Maassen, Jason; Harmelen, Frank van; Bal, Henri E.

doi:10.1016/j.websem.2011.05.004

Cited by 137 publications

(95 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the system has a high throughput: it computes the closure of about one billion LUBM triples in about 4400 seconds, which results in an input processing ratio of about 227K triples/sec. As an informal comparison, the throughput per machine of WebPIE [15] to materialize the same dataset with RDFS was about 55K triples/sec, four times lower than DynamiTE .…”

Section: Discussionmentioning

confidence: 98%

“…The design and implementation of our algorithm for rules instantiation relies on the assumption that the number of schema triples in the input is significantly smaller than the rest, so that all of them (explicit and inferred) will fit in main memory. This assumption holds for the vast majority of web data [15], but there can be scenarios where this is no longer true.…”

Section: Physical Rules Instantiationmentioning

confidence: 99%

“…In previous work, we tackled this problem proposing a distributed execution over multiple processing nodes using the MapReduce model and the Hadoop framework [15]. Schema triples were replicated on every node, and a hash join was performed against the generic triples that were being read from the input.…”

Section: Physical Rules Instantiationmentioning

confidence: 99%

“…In fact, while some of these systems can perform the materialization of several billions of statements [9,15,19], they still demand for significant computational resources, and the process may require up to a few days to complete. Because of this, most of these systems work under the assumption that the input data is static, i.e., that it does not change, or changes very infrequently.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

DynamiTE: Parallel Materialization of Dynamic RDF Data

Urbani

Margara

Jacobs

et al. 2013

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. One of the main advantages of using semantically annotated data is that machines can reason on it, deriving implicit knowledge from explicit information. In this context, materializing every possible implicit derivation from a given input can be computationally expensive, especially when considering large data volumes. Most of the solutions that address this problem rely on the assumption that the information is static, i.e., that it does not change, or changes very infrequently. However, the Web is extremely dynamic: online newspapers, blogs, social networks, etc., are frequently changed so that outdated information is removed and replaced with fresh data. This demands for a materialization that is not only scalable, but also reactive to changes. In this paper, we consider the problem of incremental materialization, that is, how to update the materialized derivations when new data is added or removed. To this purpose, we consider the ρdf RDFS fragment [11], and present a parallel system that implements a number of algorithms to quickly recalculate the derivation. In case new data is added, our system uses a parallel version of the well-known semi-naive evaluation of Datalog. In case of removals, we have implemented two algorithms, one based on previous theoretical work, and another one that is more efficient since it does not require a complete scan of the input for every update. We have evaluated the performance using a prototype system called DynamiTE , which organizes the knowledge bases with a number of indices to facilitate the query process and exploits parallelism to improve the performance. The results show that our methods are indeed capable to recalculate the derivation in a short time, opening the door to reasoning on much more dynamic data than is currently possible.

show abstract

Section: Discussionmentioning

confidence: 98%

Section: Physical Rules Instantiationmentioning

confidence: 99%

Section: Physical Rules Instantiationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

DynamiTE: Parallel Materialization of Dynamic RDF Data

Urbani

Margara

Jacobs

et al. 2013

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…The results show that quickly, over four cores, the gain of a new core is massively decreasing, against a logarithmic curve. [7] fixes some issues that optimises the reasoner implementation, improving its performance and completeness. But despite these upgrades, the results still suffer from the same issue.…”

Section: Webpiementioning

confidence: 99%

A Linked Data Reasoner in the Cloud

Chevalier

2013

The Semantic Web: Semantics and Big Data

View full text Add to dashboard Cite

Abstract. Over the last decade, the paradigm of Linked Data has gained momentum. It is possible to leverage implicit knowledge from these data using a reasoner. Nevertheless, current methods for reasoning over linked data are well suited for small to medium datasets, and they fail at reaching the scale of the Web of Data. In this PhD thesis, we are interested in how distributed computing in the Cloud can help a linked data reasoner to scale. We present in this paper the early state of this thesis.

show abstract

Tool for materializing OWL ontologies in a column‐oriented database

2018

View full text Add to dashboard Cite

Summary Big data technologies enable people to store, analyze, and utilize large amounts of complex data effectively. In this context, NoSQL (Not only Structured Query Language) databases have emerged as the most commonly used infrastructure for handling big Resource Description Framework data. In this paper, we present our proposal to materialize classified OWL (Web Ontology Language) ontologies in a NoSQL database. This materialization process enables the distributed storage of Resource Description Framework data by exploiting the inherent distribution of the NoSQL database nodes. Furthermore, this approach supposes the first step toward a novel approach to scalable OWL reasoning, which enables scalable Linked Data reasoning. To validate our proposal, we developed a prototype for a tool that stores an OWL ontology in a Cassandra database following our approach. Finally, we introduce our reasoning strategy to compute the ontology closure using this Cassandra database.

show abstract

WebPIE: A Web-scale Parallel Inference Engine using MapReduce

Cited by 137 publications

References 18 publications

DynamiTE: Parallel Materialization of Dynamic RDF Data

DynamiTE: Parallel Materialization of Dynamic RDF Data

A Linked Data Reasoner in the Cloud

Tool for materializing OWL ontologies in a column‐oriented database

Contact Info

Product

Resources

About