Approximate String Membership Checking: A Multiple Filter, Optimization-Based Approach

Abstract

Abstract-We consider the approximate string membership checking (ASMC) problem of extracting all the strings or substrings in a document that approximately match some string in a given dictionary. To solve this problem, the current state-ofart approach involves first applying an approximate, fast filter, then applying a more expensive exact verification algorithm to the strings that pass the filter. Correspondingly, many string filters have been proposed. We note that different filters are good at eliminating different strings, depending on the characteristics of the strings in both the documents and the dictionary. We suspect that no single filter will dominate all other filters everywhere. Given an ASMC problem instance and a set of string filters, we need to select the optimal filter to maximize the performance. Furthermore, in our experiments we found that in some cases a sequence of filters dominates any of the filters of the sequence in isolation, and that the best set of filters and their ordering depend upon the specific problem instance encountered. Accordingly, we propose that the approximate match problem be viewed as an optimization problem, and evaluate a number of techniques for solving this optimization problem.

GFilter: A General Gram Filter for String Similarity Search

“…Another related problem, Approximate string match, refers to the problem of matching a string or sub-string to a given pattern in a text. There have also been a lot of studies on this problem [24] [27], and Navarro gives a detailed analysis on the existing approaches in his survey [14]. The selectivity estimation of similarity search and similarity join in [3][4][5] [28].…”

Naughton's Wisconsin Bibliography: A Brief Guide

“…Related to his work on XML as well as Information Extraction, Naughton and his students worked on various problems in searching and combining textual data in databases. This includes work on combining keyword search results with forms [144,149], approximate string membership [157,160], and debugging of "why not" provenance in keyword search over databases [182].…”