A Collective, Probabilistic Approach to Schema Mapping Using Diverse Noisy Evidence

Kimmig, Angelika; Memory, Alex; Miller, Renée J.; Getoor, Lise

doi:10.1109/tkde.2018.2865785

Cited by 11 publications

(6 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given our strong positive results, we believe our metrics should become an important feature that could be used in other problems that involve understanding or integrating tables. An important open problem is to extend DomainNet to collectively resolve ambiguous metadata and data, perhaps using probabilistic graphical models that have been applied to collectively resolving multiple types of entities at once [26] and to collectively resolving data and metadata inconsistency in schema mapping [25].…”

Section: Discussionmentioning

confidence: 99%

DomainNet: Homograph Detection for Data Lake Disambiguation

Leventidis¹,

Rocco²,

Gatterbauer³

et al. 2021

Preprint

View full text Add to dashboard Cite

Modern data lakes are deeply heterogeneous in the vocabulary that is used to describe data. We study a problem of disambiguation in data lakes: how can we determine if a data value occurring more than once in the lake has different meanings and is therefore a homograph? While word and entity disambiguation have been well studied in computational linguistics, data management and data science, we show that data lakes provide a new opportunity for disambiguation of data values since they represent a massive network of interconnected values. We investigate to what extent this network can be used to disambiguate values.DomainNet uses network-centrality measures on a bipartite graph whose nodes represent values and attributes to determine, without supervision, if a value is a homograph. A thorough experimental evaluation demonstrates that state-of-the-art techniques in domain discovery cannot be re-purposed to compete with our method. Specifically, using a domain discovery method to identify homographs has a precision and a recall of 38% versus 69% with our method on a synthetic benchmark. By applying a network-centrality measure to our graph representation, DomainNet achieves a good separation between homographs and data values with a unique meaning. On a real data lake our top-200 precision is 89%.

show abstract

Section: Discussionmentioning

confidence: 99%

DomainNet: Homograph Detection for Data Lake Disambiguation

Leventidis¹,

Rocco²,

Gatterbauer³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We are currently extending Kensho in the spirit of the integration by example paradigm [78]. A large body of traditional data exchange literature is dedicated to identifying or exploiting examples that can shown to data engineers and incorporating their feedback [3,5,15,17,32,42,59,78]. Similarly, we are working on approaches for incorporating user feedback to improve upon our mapping rules.…”

Section: Discussionmentioning

confidence: 99%

Knowledge Translation: Extended Technical Report

Bashardoost,

Miller,

Lyons

et al. 2020

Preprint

View full text Add to dashboard Cite

We introduce Kensho, a tool for generating mapping rules between two Knowledge Bases (KBs). To create the mapping rules, Kensho starts with a set of correspondences and enriches them with additional semantic information automatically identified from the structure and constraints of the KBs. Our approach works in two phases. In the first phase, semantic associations between resources of each KB are captured. In the second phase, mapping rules are generated by interpreting the correspondences in a way that respects the discovered semantic associations among elements of each KB. Kensho's mapping rules are expressed using SPARQL queries and can be used directly to exchange knowledge from source to target. Kensho is able to automatically rank the generated mapping rules using a set of heuristics. We present an experimental evaluation of Kensho and assess our mapping generation and ranking strategies using more than 50 synthesized and real world settings, chosen to showcase some of the most important applications of knowledge translation. In addition, we use three existing benchmarks to demonstrate Kensho's ability to deal with different mapping scenarios.

show abstract

“…There are numerous prior-arts in schema matching [29], [38], [51], [72], which mainly match schemas based on metadata (e.g., attribute name) and/or instances. Entity matching (EM) [16], which is to identify data instances that refer to the same real-world entity, is also related.…”

Section: Schema/entity Matchingmentioning

confidence: 99%

Survive the Schema Changes: Integration of Unmanaged Data Using Deep Learning

Wang,

Zhou,

Das

et al. 2020

Preprint

View full text Add to dashboard Cite

Data is the king in the age of AI. However data integration is often a laborious task that is hard to automate. Schema change is one significant obstacle to the automation of the end-to-end data integration process. Although there exist mechanisms such as query discovery and schema modification language to handle the problem, these approaches can only work with the assumption that the schema is maintained by a database. However, we observe diversified schema changes in heterogeneous data and open data, most of which has no schema defined. In this work, we propose to use deep learning to automatically deal with schema changes through a super cell representation and automatic injection of perturbations to the training data to make the model robust to schema changes. Our experimental results demonstrate that our proposed approach is effective for two realworld data integration scenarios: coronavirus data integration, and machine log integration.

show abstract

A Collective, Probabilistic Approach to Schema Mapping Using Diverse Noisy Evidence

Cited by 11 publications

References 38 publications

DomainNet: Homograph Detection for Data Lake Disambiguation

DomainNet: Homograph Detection for Data Lake Disambiguation

Knowledge Translation: Extended Technical Report

Survive the Schema Changes: Integration of Unmanaged Data Using Deep Learning

Contact Info

Product

Resources

About