An Iterative, Self-Assessing Entity Resolution System: First Steps toward a Data Washing Machine

Talburt, John R.; Sarkhi, Awaad K. Al; Pullen, Daniel L.; Claassens, Leon; Wang, Thomas J.

doi:10.14569/ijacsa.2020.0111279

Cited by 13 publications

(16 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here we developed a solution that relies on exploiting the transitivity property characterized by the output of the matching process, allowing us to recast the matched documents as a graph of weighted edges. We expand on the work that our group has done, mainly the Graph-based Data Washing Machine (GDWM) [15] and the original Data Washing Machine (DWM) [1]. We address the previously mentioned problems as follows:…”

Section: A Contributionmentioning

confidence: 99%

“…Entity resolution is critical in data cleaning, curation, and integration [1]. It also refers to finding duplicate records within the same table, across various tables, or multiple databases that might refer to the same entity.…”

Section: Introductionmentioning

confidence: 99%

“…The clustering process aims to resolve conflicts in pairwise matching and find records that indirectly match. Those conflicts typically occur due to the reliance on frequency-based blocking [1]. Thus, to increase automation, reduce the amount of human input, and increase efficiency in the unsupervised entity resolution process, we aim to reduce the number of input parameters needed and to step away from direct approaches in approximate string matching.…”

Section: Introductionmentioning

confidence: 99%

“…More specifically, we address the following challenges in graph-based unsupervised entity resolution systems represented by [13] and iterative self-assessing systems represented by [1]:…”

Section: Introductionmentioning

confidence: 99%

“…• Setting matching thresholds as in [13], and cluster quality thresholds as in [1] might be problematic for the user's perspective. Interpreting those thresholds depends on the fed data, and the user might not have a baseline reference to compare.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

ModER: Graph-based Unsupervised Entity Resolution using Composite Modularity Optimization and Locality Sensitive Hashing

Ebeid¹,

Talburt²,

Hagan³

et al. 2022

IJACSA

View full text Add to dashboard Cite

Entity resolution describes techniques used to identify documents or records that might not be duplicated; nevertheless, they might refer to the same entity. Here we study the problem of unsupervised entity resolution. Current methods rely on human input by setting multiple thresholds prior to execution. Some methods also rely on computationally expensive similarity metrics and might not be practical for big data. Hence, we focus on providing a solution, namely ModER, capable of quickly identifying entity profiles in ambiguous datasets using a graphbased approach that does not require setting a matching threshold. Our framework exploits the transitivity property of approximate string matching across multiple documents or records. We build on our previous work in graph-based unsupervised entity resolution, namely the Data Washing Machine (DWM) and the Graphbased Data Washing Machine (GDWM). We provide an extensive evaluation of a synthetic data set. We also benchmark our proposed framework using state-of-the-art methods in unsupervised entity resolution. Furthermore, we discuss the implications of the results and how it contributes to the literature.

show abstract

Section: A Contributionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…More specifically, we address the following challenges in graph-based unsupervised entity resolution systems represented by [13] and iterative self-assessing systems represented by [1]:…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations