Efficient randomized pattern-matching algorithms

Abstract

We present randomized algorithms to solve the following string-matching problem and some of its generalizations: Given a string X of length n (the pattern) and a string Y (the text), find the first occurrence of X as a consecutive block within Y. The algorithms represent strings of length n by much shorter strings called fingerprints, and achieve their efficiency by manipulating fingerprints instead of longer strings. The algorithms require a constant number of storage locations, and essentially run in real time. They are conceptually simple and easy to implement. The method readily generalizes to higher-dimensional patternmatching problems. IntroductionText-processing systems must allow their users to search for a given character string within a body of text. Database systems must be capable of searching for records with stated values in specified fields. Such problems are instances of the following string-matching problem: For a specified set {{X{i), Y(i)}\ of pairs of strings, determine, if possible, an r such that X(r) = Y(r). Usually the set is sjiecified not by explicit enumeration of the pairs, but rather by a rule for computing the pairs (X(i), Y{i)} from some given data. *<^opjTight 1987 by International Business Machines Corporation. Copying in printed form for private use is permitted without payment of royalty provided that (1) each reproduction is done without alteration and (2) the Journal reference and IBM copyright notice are included on the first page. The title and abstract, but no other portions, of this paper may be copied or distributed royalty free without further permission by computer-based and other information-service systems. Permission to republish any other portion of this paper must be obtained from the Editor.We present a randomized algorithm to solve this problem. The algorithm associates with each string X a fingerprint <t>{X) which is much shorter than the string itself The search for a match then compares short fingerprints instead of long strings.The algorithm selects the fingerprint function <t> at random from a family of easy-to-compute functions. No matter which input \{X(i), F(/)>) is presented, the algorithm is unlikely to produce afiilse match, in which two fingerprints agree even though the original strings do not.The most widely studied pattern-matching problem is the following: Given a pattern X of length n and a text Y of length m'>n, find the first occurrence of X as a consecutive substring of Y. Several linear time algorithms have been given for this problem. The algorithms of Knuth, Morris, and Pratt [ 1 ] (KMP in the sequel) and of Boyer and Moore [2] require, for fast implementation, 0{n) registers to store a table of pyointers. The characters of the text Y can come in a stream and require no storage. But for fast implementation it is useful to have portions of Y in main memory. The algorithm of Galil and Seiferas [3] requires only C>(log n) registers. Recently Galil and Seiferas [4] have found a realtime algorithm using a constant number of registers.Our method,...

5PM: Secure Pattern Matching

“…It is used in many areas, including text processing, database search [1], networking and security applications [2] and recently in the context of bioinformatics and DNA analysis [3,4,5]. It is a problem that has been extensively studied, resulting in several efficient (although insecure) techniques to solve its many variations, e.g., [6,7,8,9]. The most common interpretation of the pattern matching problem is the following: given a finite alphabet Σ, a text T ∈ Σ n and a pattern p ∈ Σ m , the exact pattern matching decision problem requires one to decide whether or not a pattern appears in the text.…”

More Than 1700 Years of Word Equations

“…Moreover, these matrices generate exactly those matrices on SL(2, Z) where all coefficients are natural numbers. This is actually easy to show, and also used in fast "fingerprint" pattern matching algorithm by Karp and Rabin [12]. A reduction from WordEquations to Hilbert 10 is now straightforward.…”

Parallel String Matching

“…Prominent examples are Horspool [15] and QuickSearch [25], simplifying variations of Boyer-Moore, which have proven to be efficient in practice. The Rabin-Karp [16] algorithm is an alternative solution to the string matching problem, testing for matches based on hashes computed from the input text and pattern.…”