Automatic generation of efficient string matching algorithms by generalized partial computation

Abstract

This paper shows that Generalized Partial Computation (GPC) can automatically generate efficient string matching algorithms. GPC is a program transformation method utilizing partial information about input data and auxiliary functions as well as the logical structure of a source program. GPC uses both a classical partial evaluator and an inference engine such as a theorem prover to optimize programs. First, we show that a Boyer-Moore (BM) type pattern matcher without the bad-character heuristic can be generated from a simple non-linear backward matcher by GPC. This sort of problems has already been discussed in the literature using offline partial evaluators. However, there was no proof that every generated matcher runs in the same way as the BM. In this paper we prove that the problem can be solved starting from a simple non-linear pattern matcher as a source program. We also prove that a Knuth-Morris-Pratt (KMP) type linear string matcher can be generated from a naive non-linear forward matcher by GPC.

Measuring the Propagation of Information in Partial Evaluation

“…It is also the most popular Boyer-Moore variant in the program generation community [7,16,26] 3 2 4 6 8 10 9 8 7 |pattern| Original GS 3 2 4 5 4 6 7 6 8 9 8 10 9 8 7 |pattern| The Automaton group Finally, we consider the Automaton and other matchers that rely solely on table comparisons. Since no negative information is ever generated, such matchers probe each text character at most once as long as positive information is not pruned.…”

Fast Partial Evaluation of Pattern Matching in Strings

“…To obtain this effect, one can either rewrite the matcher so that backtracking only depends on static data (such a rewriting is known as a binding-time improvement or a staging transformation [34]) and use a simple partial evaluator [5,15], or keep the matcher as is and use an advanced partial evaluator [18,37,38,40]. In this article, the starting point is a staged quadratic-3 time matcher and a simple memoizing partial evaluator, such as Similix, where specialization points are dynamic conditional expressions [8].…”

“…Examples of the methods used are Dijkstra's invariants [17], Bird's recursion introduction and tabulation [7], Takeichi and Akama's equational reasoning [41], Colussi's Hoare logic [14], and Hernández and Rosenblueth's logicprogram derivation [25]. Variants of the Boyer-Moore have also been reconstructed: both by partial evaluation [4,18], and by logic-program derivation [24].…”

“…In contrast, a partial evaluator does not specialize a string matcher in linear time. This shortcoming was already stated in Consel and Danvy's first report of a solution [15] and it has been mentioned ever since, up to and including Futamura's keynote speech at ASIA-PEPM 2002 [18]. The second half of the problem could therefore serve as a catalyst for the development of fast partial evaluators.…”

“…Given a quadratic string matcher that searches for the first occurrence of a pattern in a text, a partial evaluator specializes this string matcher with respect to a pattern and yields a residual program that traverses the text in linear time. The problem was first stated by Yoshihiko Futamura in 1987 [20] and since then, it has served as a catalyst for the development of partial evaluators, giving rise to a variety of solutions [3,4,15,18,19,20,22,26,32,35,38,40,41].…”