Data Enrichment Toolchain: A Data Linking and Enrichment Platform for Heterogeneous Data

Abstract

Proliferation of data sources associated to Internet of Things (IoT) deployment as well as those bound to Open Data Portals (e.g. European Data Portal, Municipalities Open Data Portals, etc.) and Social Media platforms is creating an abundance of information that is called to bring benefits for both the private and public sectors, through the development of added-value services, increasing administrations' transparency and availability or fostering efficiency of public services. However, pieces of information without a context are significantly less valuable. Raw data lacks semantics and it is highly heterogeneous from one data-source to another. This poses a challenge to make it useful. To turn all this data into valuable information it is necessary to enable its combination so that meaningful context can be created. Moreover, it is fundamental to define the mechanisms enabling the adoption and orchestration of advanced (typically AIenabled) data processing techniques to be applied over the harmonized datasets and data-streams. This paper presents the Data Enrichment Toolchain (DET) that provides the necessary harmonization and enrichment to datasets and data-streams coming from heterogeneous sources. The value of the enriched data lies on the one hand in the transfer of the data into a semantically grounded knowledge graph and, on the other hand, in the creation of new data through linking, aggregating and reasoning on the data. In both cases, the benefit of employing linked-data modelling and semantics comes from the extension of the metadata that is associated to every piece of information. Furthermore, the experimental evaluation of the DET implementation that we have carried out is also presented in the paper.

Reshaping Smart Cities through NGSI-LD Enrichment

“…Amidst these initiatives, the DET presented in [ 10 ] emerges as a complete and integrated solution. This work offers a set of tools that effectively enables information enrichment across various application domains.…”

“…In addressing the intricacies of smart cities, our linkers and enrichers stand as practical solutions rooted in the challenges and limitations discussed in [ 24 , 25 ]. Within the realm of interoperability, we chose to found our work in the framework provided by [ 10 ]. Our commitment to utilising the complete range of interoperability capabilities motivated this decision, guaranteeing that our linkers and enrichers function within a standardised and adaptable framework able to homogenise data from domains of diverse nature.…”

“…The focus of this article is twofold: firstly, to examine the complexities involved in utilising the data enrichment toolchain (DET) presented in [ 10 ] as a versatile framework to bridge the aforementioned gaps, and secondly, to analyse the implementation of several data processing modules and the resulting high-value data. The DET acts as a catalyst for implementing data enrichers and linkers tailored to meet the specific demands of smart cities.…”

A Connector for Integrating NGSI-LD Data into Open Data Portals

“…This section outlines the specific use case that has been employed for the implementation and validation of the modules described above. The NGSI-LD-based domain used for the validation was the Data Enrichment Toolchain (DET) implemented and deployed within the Situation-Aware Linked heTerogeneous Enriched Data (SALTED) project [ 33 ]. This toolchain harmonises and enriches heterogeneous data by applying the principles of linked data and semantics, with the aim of achieving the so-essential interoperability.…”