Profiling and mining RDF data with ProLOD++

Abstract

Before reaping the benefits of open data to add value to an organizations internal data, such new, external datasets must be analyzed and understood already at the basic level of data types, constraints, value patterns etc. Such data profiling, already difficult for large relational data sources, is even more challenging for RDF datasets, the preferred data model for linked open data.We present ProLOD++, a novel tool for various profiling and mining tasks to understand and ultimately improve open RDF data. ProLOD++ comprises various traditional data profiling tasks, adapted to the RDF data model. In addition, it features many specific profiling results for open data, such as schema discovery for user-generated attributes, association rule discovery to uncover synonymous predicates, and uniqueness discovery along ontology hierarchies. ProLOD++ is highly efficient, allowing interactive profiling for users interested in exploring the properties and structure of yet unknown datasets. I. PROFILING LINKED OPEN DATAAt the time of writing, Linked Open Data (LOD) as compiled in http://linkeddata.org comprised already more than 300 data sources including prominent examples, such as DBpedia, YAGO, and Freebase. A LOD dataset is usually represented in the Resource Description Framework (RDF) embodying an entity-relationship-graph or a set of triplified facts consisting of subjects, predicates, and objects. Most of the datasets are openly available and connected amongst each other via sameAs links between representations of same real-world entities. Hundreds more open RDF datasets are listed for instance at http://datahub.io.However, consuming LOD is not easy, because the sources are heterogeneous, often inconsistent, and lack often even basic metadata. One of the main reasons for this problem is that many of the data sources, such as DBpedia [7] or YAGO [13], have been extracted from unstructured data. Furthermore, a knowledge base usually evolves over time when more facts and entities are added and rigid schema and ontology definitions, hand-crafted at some point of time, lose validity over all entities of the dataset. Hence it is vital to thoroughly examine and understand each dataset, its structure, and its properties before usage.Manually inspecting datasets can achieve this goal only to a limited extent: algorithms and tools are needed that profile the dataset to retrieve relevant and interesting meta-data analyzing the entire dataset [14]. Indeed, there are many commercial tools, such as IBM's Information Analyzer, Microsoft's SQL Server Integration Services (SSIS), or Informatica's Data Explorer, and some research prototypes, such as [12], for profiling relational datasets. However all of these tool were designed to profile relational data. LOD which is represented in RDF data has a very different nature and calls for specific profiling and mining techniques. Current tools to work on RDF data are limited to graph visualization and editing: LODlive 1 is a browser-based tool to browse and search in RDF datasets. RDF Pro...

An Effective and Efficient MapReduce Algorithm for Computing BFS-Based Traversals of Large-Scale RDF Graphs

“…We have also provided experiments that clearly demonstrate the speed-up ensured by our solutions over comparative approaches. This offers exciting possibilities towards the support of RDF graph analysis and mining methodologies (e.g., [80,81]) over large-scale data sets, thanks to the powerful run-time support offered by MapReduce.…”

Profiling relational data: a survey

“…Another example of clustering in the context of data profiling is ProLOD++, which applies k-means clustering to Rdf relations [1]. We refer the reader to surveys by Jain et al [78] and Xu and Wunsch II [137] for more details on clustering algorithms for relational data.…”

“…Most data cleansing tools can then either transform differently formatted numbers or mark them as improper. 1 See Sect. 6 for a more comprehensive list of tools.…”

“…1 These approaches follow the same general procedure: A user specifies the data to be profiled and selects the types of metadata to be generated. Next, the tool computes the metadata in batch mode, using SQL queries and/or specialized algorithms.…”

“…ProLod++ provides additional functionalities by applying clustering and rule mining techniques [1]. When profiling Rdf data, there are many interesting metadata beyond simple statistics and patterns of Rdf statement elements, including synonymously used properties [4], inverse relationships of properties, the conformity and consistence of Rdf structured data to the corresponding ontology [2], and the distribution of literals and de-referenceable resource Uris from different namespaces.…”