Efficient Merging and Filtering Algorithms for Approximate String Searches

Li, Chen; Lu, Jiaheng; Lu, Yiming

doi:10.1109/icde.2008.4497434

Cited by 229 publications

(277 citation statements)

References 7 publications

Supporting

Mentioning

269

Contrasting

Unclassified

Order By: Relevance

“…Progressive computation is orthogonal to the filtering techniques and we do not consider progressive computation in comparing different filters in our work. Different filters have been used in an ad hoc fashion to accelerate the membership checking, e.g., the length filter is used in [19], [20]. However, none of the previous work exploits the many existing string filters and uses them systematically.…”

Section: Related Workmentioning

confidence: 99%

“…In [19], efficient exact algorithms are proposed to conduct approximate string checking based on merging token inverted lists. In [6], the ISH (Inverted Signature-based Hashtable) structure is presented with the focus on reducing the filter checking time.…”

Section: Related Workmentioning

confidence: 99%

“…Approximate string membership checking ( [6], [18], [19], [20]) is another variation on the approximate string matching problem, in which we view one string collection as the dictionary and process a second collection of strings (or documents) to check if each string in the second collection approximately matches some dictionary string. In [19], efficient exact algorithms are proposed to conduct approximate string checking based on merging token inverted lists.…”

Section: Related Workmentioning

confidence: 99%

“…We implemented the DivideSkip algorithm [19] to merge the inverted lists of tokens for candidate strings.…”

Section: Ish Filter (4)mentioning

confidence: 99%

See 3 more Smart Citations

Approximate String Membership Checking: A Multiple Filter, Optimization-Based Approach

Sun

Naughton

Barman

2012

2012 IEEE 28th International Conference on Data Engineering

View full text Add to dashboard Cite

Abstract-We consider the approximate string membership checking (ASMC) problem of extracting all the strings or substrings in a document that approximately match some string in a given dictionary. To solve this problem, the current state-ofart approach involves first applying an approximate, fast filter, then applying a more expensive exact verification algorithm to the strings that pass the filter. Correspondingly, many string filters have been proposed. We note that different filters are good at eliminating different strings, depending on the characteristics of the strings in both the documents and the dictionary. We suspect that no single filter will dominate all other filters everywhere. Given an ASMC problem instance and a set of string filters, we need to select the optimal filter to maximize the performance. Furthermore, in our experiments we found that in some cases a sequence of filters dominates any of the filters of the sequence in isolation, and that the best set of filters and their ordering depend upon the specific problem instance encountered. Accordingly, we propose that the approximate match problem be viewed as an optimization problem, and evaluate a number of techniques for solving this optimization problem.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…We implemented the DivideSkip algorithm [19] to merge the inverted lists of tokens for candidate strings.…”

Section: Ish Filter (4)mentioning

confidence: 99%

See 2 more Smart Citations

Approximate String Membership Checking: A Multiple Filter, Optimization-Based Approach

Sun

Naughton

Barman

2012

2012 IEEE 28th International Conference on Data Engineering

View full text Add to dashboard Cite

show abstract

“…Several existing work studies the similarity search problem [24], [25], [26], [27], which returns the records in a collection whose similarity with the query exceeds a given threshold. Based on the inverted list framework, [26] proposes an efficient principle to skip records when accessing inverted lists.…”

Section: Related Workmentioning

confidence: 99%

Top-k Set Similarity Joins

Xiao

Wang

Lin

et al. 2009

2009 IEEE 25th International Conference on Data Engineering

158

106

View full text Add to dashboard Cite

Abstract-Similarity join is a useful primitive operation underlying many applications, such as near duplicate Web page detection, data integration, and pattern recognition. Traditional similarity joins require a user to specify a similarity threshold. In this paper, we study a variant of the similarity join, termed top-k set similarity join. It returns the top-k pairs of records ranked by their similarities, thus eliminating the guess work users have to perform when the similarity threshold is unknown before hand. An algorithm, topk-join, is proposed to answer top-k similarity join efficiently. It is based on the prefix filtering principle and employs tight upper bounding of similarity values of unseen pairs. Experimental results demonstrate the efficiency of the proposed algorithm on large-scale real datasets.

show abstract

Searchable encryption: A survey on privacy‐preserving search schemes on encrypted outsourced data

Handa

Krishna

Aggarwal

2019

Concurrency and Computation

View full text Add to dashboard Cite

Outsourcing confidential data to cloud storage leads to privacy challenges that can be reduced using encryption. However, with encryption in place, the utilization of the data is reduced, which leads to reduced quality of experience of the users. To overcome this, searchable encryption (SE) schemes are utilized, which allow the end users to retrieve the relevant documents from the cloud, for which various researchers have worked utilizing different techniques. Despite the popularity of the searchable encryption schemes, most of the surveys either do not provide or present an incomplete taxonomy of SE schemes. Hence, in this paper, we attempt to present a complete taxonomy/classification of the searchable encryption schemes in terms of the type of search, type of index, results retrieved, implementation type, multiplicity of users, and the technique used. From the literature, it is observed that inner product similarity is widely adopted by researchers to compute the similarity of the query and the document index as it provides both conjunctive and disjunctive searching (ie, have better search capability) but requires high search time (ie, have lower search efficiency). On the other hand, schemes based on binary comparisons exist, which require less search time (ie, have better search efficiency) but support only conjunctive searching (ie, have limited search capability). Thus, a major conclusion drawn from our work is that there is an imbalance between search capability and search efficiency, ie, in the existing schemes, search capability can be improved at the cost of search time only. Therefore, we suggest that one direction where researchers should work on is to provide a balance between search capability and search efficiency.

show abstract

Efficient Merging and Filtering Algorithms for Approximate String Searches

Cited by 229 publications

References 7 publications

Approximate String Membership Checking: A Multiple Filter, Optimization-Based Approach

Approximate String Membership Checking: A Multiple Filter, Optimization-Based Approach

Top-k Set Similarity Joins

Searchable encryption: A survey on privacy‐preserving search schemes on encrypted outsourced data

Contact Info

Product

Resources

About