The Four Generations of Entity Resolution

Papadakis, George; Ioannou, Ekaterini; Thanos, Emanouil; Palpanas, Themis

doi:10.1007/978-3-031-01878-7

Cited by 52 publications

(8 citation statements)

References 237 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Entity resolution: Entity resolution is a process that is typically divided into two steps: blocking and matching [19]. The blocking step groups similar records into clusters to reduce the computational cost of entity resolution, while the matching step involves comparing records within the same cluster to determine whether they match.…”

Section: Related Workmentioning

confidence: 99%

PromptER: Prompt Contrastive Learning for Generalized Entity Resolution

Dai,

Tang,

et al. 2024

Artificial Intelligence and Applications

View full text Add to dashboard Cite

Entity resolution (ER), which aims to identify whether data records from various sources refer to the same real-world entity, is a crucial part of data integration systems. Traditional ER solutions assumes that data records are stored in relational tables with an aligned schema. However, in practical applications, it is common that data records to be matched may have different formats (e.g., relational, semi-structured, or textual types). In order to support ER for data records with varying formats, Generalized Entity Resolution has been proposed and has recently gained much attention. In this paper, we propose PromptER, a model based on pre-trained language models that offers an efficient and effective approach to accomplish Generalized Entity Resolution tasks. PromptER starts with a supervised contrastive learning process to train a Transformer encoder, which is afterward used for blocking and fine-tuned for matching. Specially, in the record embedding process, PromptER uses the proposed prompt embedding technique to better utilized the pre-trained language model layers and avoid embedding bias. Morever, we design a novel data augmentation method and an evaluation method to enhance the performance of the proposed model. We conduct experiments on the Generalized Entity Resolution dataset Machamp and the results show that PromptER significantly outperforms other state-of-art methods in the blocking and matching tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

PromptER: Prompt Contrastive Learning for Generalized Entity Resolution

Dai,

Tang,

et al. 2024

Artificial Intelligence and Applications

View full text Add to dashboard Cite

show abstract

“…There is a rich body of literature on ER [3,8,9,46]. Following the seminal Fellegi-Sunter model for record linkage [16], a major focus of prior work has been on classifying pairs of input records as match, non-match, or potential match.…”

Section: Related Workmentioning

confidence: 99%

An analysis of one-to-one matching algorithms for entity resolution

et al. 2023

Self Cite

View full text Add to dashboard Cite

Entity resolution (ER) is the task of finding records that refer to the same real-world entities. A common scenario, which we refer to as Clean-Clean ER, is to resolve records across two clean sources (i.e., they are duplicate-free and contain one record per entity). Matching algorithms for Clean-Clean ER yield bipartite graphs, which are further processed by clustering algorithms to produce the end result. In this paper, we perform an extensive empirical evaluation of eight bipartite graph matching algorithms that take as input a bipartite similarity graph and provide as output a set of matched records. We consider a wide range of matching algorithms, including algorithms that have not previously been applied to ER, or have been evaluated only in other ER settings. We assess the relative performance of these algorithms with respect to accuracy and time efficiency over ten established real-world data sets, from which we generated over 700 different similarity graphs. Our results provide insights into the relative performance of these algorithms and guidelines for choosing the best one, depending on the data at hand.

show abstract

“…Nevertheless, we are far from solving the problem at human performance levels, and errors can be costly. While initially (and in some contexts still), rule-based and manually engineered solutions were prevalent [47,67], machine learning methods became increasingly popular as ER solutions, as with many other problems amenable to learning from data, starting from the early-mid 2000s [63,32]. Over the last decade, deep learning solutions have also been applied to ER [16,15,43,31,35], and even more recently, transformer-based models such as BERT [38].…”

Section: Introductionmentioning

confidence: 99%

Cost-Efficient Prompt Engineering for Unsupervised Entity Resolution

Nananukul,

Sisaengsuwanchai,

Kejriwal

2024

Preprint

View full text Add to dashboard Cite

Entity Resolution (ER) is the problem of semi-automatically determining when two entities refer to the same \emph{underlying} entity, with applications ranging from healthcare to e-commerce. Traditional ER solutions required considerable manual expertise, including domain-specific feature engineering, as well as identification and curation of training data. Recently released large language models (LLMs) provide an opportunity to make ER more seamless and domain-independent. However, it is also well known that LLMs can pose risks, and that the quality of their outputs can depend on how prompts are engineered. Unfortunately, a systematic experimental study on the effects of different prompting methods for addressing unsupervised ER, using LLMs like ChatGPT, has been lacking thus far. This paper aims to address this gap by conducting such a study. We consider some relatively simple and cost-efficient ER prompt engineering methods and apply them to ER on two real-world datasets widely used in the community. We use an extensive set of experimental results to show that an LLM like GPT3.5 is viable for high-performing unsupervised ER, and interestingly, that more complicated and detailed (and hence, expensive) prompting methods do not necessarily outperform simpler approaches. We provide brief discussions on qualitative and error analysis, including a study of the inter-consistency of different prompting methods to determine whether they yield stable outputs. Finally, we consider some limitations of LLMs when applied to ER.

show abstract

The Four Generations of Entity Resolution

Cited by 52 publications

References 237 publications

PromptER: Prompt Contrastive Learning for Generalized Entity Resolution

PromptER: Prompt Contrastive Learning for Generalized Entity Resolution

An analysis of one-to-one matching algorithms for entity resolution

Cost-Efficient Prompt Engineering for Unsupervised Entity Resolution

Contact Info

Product

Resources

About