Bootstrapping pay-as-you-go data integration systems

Abstract

Data integration systems offer a uniform interface to a set of data sources. Despite recent progress, setting up and maintaining a data integration application still requires significant upfront effort of creating a mediated schema and semantic mappings from the data sources to the mediated schema. Many application contexts involving multiple data sources (e.g., the web, personal information management, enterprise intranets) do not require full integration in order to provide useful services, motivating a pay-as-you-go approach to integration. With that approach, a system starts with very few (or inaccurate) semantic mappings and these mappings are improved over time as deemed necessary.This paper describes the first completely self-configuring data integration system. The goal of our work is to investigate how advanced of a starting point we can provide a pay-as-you-go system. Our system is based on the new concept of a probabilistic mediated schema that is automatically created from the data sources. We automatically create probabilistic schema mappings between the sources and the mediated schema. We describe experiments in multiple domains, including 50-800 data sources, and show that our system is able to produce high-quality answers with no human intervention.

An Uncertain Data Integration System

“…they generate only one single deterministic mediated schema. We also compare our system with the one presented in [4] which is the closest to ours.…”

“…For building such a system, we take advantage of the background knowledge which is implied in the functional dependencies (FDs) defined on the schemas. Since probabilistic data models have shown to be promising [3][4][5], we build our approach on a probabilistic data model to capture the uncertainty which arises during the schema matching process. Therefore, we generate a set of Probabilistic Mediated Schemas (PMSs).…”

“…In the database literature, the closest related work to ours is that of Sarma et al [4] which based on PMSs proposed UDI (Uncertain Data Integration), an uncertain data integration system. However, UDI may fail to capture some important attribute correlations, and thereby produce low quality answers.…”

“…However, UDI may fail to capture some important attribute correlations, and thereby produce low quality answers. Let us clarify this by an example which is the same as the running example in [4].…”

Ten Challenges for Ontology Matching

“…Finally, the work in [16] introduced the notion of probabilistic schema mappings (correspondences), namely a set of mappings with a probability attached to each mapping; and, used it to answer queries with uncertainty about semi-automatically created mappings. Imprecise mappings can be further improved over time as deemed necessary, for example, within the settings of approximate data integration, see, e.g., [72].…”