Incorporating string transformations in record matching

Arasu, Arvind; Chaudhuri, Surajit; Ganjam, Kris; Kaushik, Raghav

doi:10.1145/1376616.1376742

Cited by 13 publications

(12 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We ran the algorithm on all the datasets with different lp values. Figures 2(c)-2(f) show various measures on the DBLP dataset for τ ∈ [1,3]. Results on other datasets are similar.…”

Section: Effect Of Prefix Lengthmentioning

confidence: 63%

“…A recent experimental study of their relative effectiveness is presented in [9]. Some new types of record recently studied include utilizing group information [30], combining multiple similarity functions [11], leveraging aggregate constraints [13], and considering string transformation rules [3].…”

Section: Related Workmentioning

confidence: 99%

“…3 An anecdotal example is that we are able to Ronald Regan to Ronald Reagan in the CONLL dataset used in the experiment.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Efficient approximate entity extraction with edit distance constraints

Wang

Xiao

Lin

et al. 2009

Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

Named entity recognition aims at extracting named entities from unstructured text. A recent trend of named entity recognition is finding approximate matches in the text with respect to a large dictionary of known entities, as the domain knowledge encoded in the dictionary helps to improve the extraction performance.In this paper, we study the problem of approximate dictionary matching with edit distance constraints. Compared to existing studies using token-based similarity constraints, our problem definition enables us to capture typographical or orthographical errors, both of which are common in entity extraction tasks yet may be missed by token-based similarity constraints. Our problem is technically challenging as existing approaches based on q-gram filtering have poor performance due to the existence of many short entities in the dictionary. Our proposed solution is based on an improved neighborhood generation method employing novel partitioning and prefix pruning techniques. We also propose an efficient document processing algorithm that minimizes unnecessary comparisons and enumerations and hence achieves good scalability. We have conducted extensive experiments on several publicly available named entity recognition datasets. The proposed algorithm outperforms alternative approaches by up to an order of magnitude.

show abstract

“…We ran the algorithm on all the datasets with different lp values. Figures 2(c)-2(f) show various measures on the DBLP dataset for τ ∈ [1,3]. Results on other datasets are similar.…”

Section: Effect Of Prefix Lengthmentioning

confidence: 63%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Efficient approximate entity extraction with edit distance constraints

Wang

Xiao

Lin

et al. 2009

Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

show abstract

“…We compare with them in more detail in the next subsection. The learned rules can be used to identify more coreferent pairs [18,19].…”

Section: Relatedmentioning

confidence: 99%

“…Lines 12-13 calculate the scores for each candidate rule. Then we iteratively select the TopRule which has the maximum score (Lines [16][17][18][19][20], and withdraw support from other conflicting candidate rules by the procedure UpdateRules. We repeat this process until k high-quality rules are found.…”

Section: Select the Top-k Rulesmentioning

confidence: 99%

Learning Top-k Transformation Rules

Patro

Wang

2011

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Record linkage identifies multiple records referring to the same entity even if they are not bit-wise identical. It is thus an essential technology for data integration and data cleansing. Existing record linkage approaches are mainly relying on similarity functions based on the surface forms of the records, and hence are not able to identify complex coreference records. This seriously limits the effectiveness of existing approaches. In this work, we propose an automatic method to extract top-k high quality transformation rules given a set of possibly coreferent record pairs. We propose an effective algorithm that performs careful local analyses for each record pair and generates candidate rules; the algorithm finally chooses top-k rules based on a scoring function. We have conducted extensive experiments on real datasets, and our proposed algorithm has substantial advantage over the previous algorithm in both effectiveness and efficiency.

show abstract

Scalable Processing of Spatial-Keyword Queries

Mahmood

Aref

2019

Synthesis Lectures on Data Management

View full text Add to dashboard Cite

Incorporating string transformations in record matching

Cited by 13 publications

References 4 publications

Efficient approximate entity extraction with edit distance constraints

Efficient approximate entity extraction with edit distance constraints

Learning Top-k Transformation Rules

Scalable Processing of Spatial-Keyword Queries

Contact Info

Product

Resources

About