Evaluating entity resolution results

Abstract

Entity Resolution (ER) is the process of identifying groups of records that refer to the same real-world entity. Various measures (e.g., pairwise F1, cluster F1) have been used for evaluating ER results. However, ER measures tend to be chosen in an ad-hoc fashion without careful thought as to what defines a good result for the specific application at hand. In this paper, our contributions are twofold. First, we conduct an analysis on existing ER measures, showing that they can often conflict with each other by ranking the results of ER algorithms differently. Second, we explore a new distance measure for ER (called "generalized merge distance" or GM D) inspired by the edit distance of strings, using cluster splits and merges as its basic operations. A significant advantage of GM D is that the cost functions for splits and merges can be configured, enabling us to clearly understand the characteristics of a defined GM D measure. Surprisingly, a state-of-the-art clustering measure called Variation of Information is a special case of our configurable GM D measure, and the widely used pairwise F1 measure can be directly computed using GM D. We present an efficient lineartime algorithm that correctly computes the GM D measure for a large class of cost functions that satisfy reasonable properties.

Populating a linked data entity name system

“…An alternative to a precision-recall curve is Receiver Operating Characteristic (ROC), which plots true positives against false positives (Hanley & McNeil, 1982). Historically, and currently, precision-recall curves dominate ROC curves in the instance matching community (Menestrina, Whang & Garcia-Molina, 2010;Köpcke, Thor & Rahm, 2010). In keeping with existing trends in the literature, precision-recall curves are favored over ROC curves for similarity evaluations in this dissertation.…”

Qasca

“…Note that there are other metrics defined based on different purposes. In entity resolution, there are also several cluster-based metrics, such as K-measure [27], GM D measure [36] and Rand-index [49]. For strings, similarity-based metrics including Jaccard, Dice, Cosine and Edit Distances are defined and used [11,55,58].…”

Adaptive Windows for Duplicate Detection

“…To evaluate duplicate detection results, a variety of evaluation metrics exists [21,22]. As we want to evaluate algorithms that select candidate pairs, we do not use a similarity function in Sec.…”