A Discriminative Model of Stochastic Edit Distance in the Form of a Conditional Transducer

Abstract

Abstract. Many real-world applications such as spell-checking or DNA analysis use the Levenshtein edit-distance to compute similarities between strings. In practice, the costs of the primitive edit operations (insertion, deletion and substitution of symbols) are generally hand-tuned. In this paper, we propose an algorithm to learn these costs. The underlying model is a probabilitic transducer, computed by using grammatical inference techniques, that allows us to learn both the structure and the probabilities of the model. Beyond the fact that the learned transducers are neither deterministic nor stochastic in the standard terminology, they are conditional, thus independant from the distributions of the input strings. Finally, we show through experiments that our method allows us to design cost functions that depend on the string context where the edit operations are used. In other words, we get kinds of context-sensitive edit distances.

Learning Constrained Edit State Machines

“…In the context of probabilistic machines, the maximization of the likelihood is often used. In this paper, we follow the same idea that explains why we are interested in learning string edit similarities in a probabilistic context rather than learning a true edit metric 1 . In our approach, we aim to learn a conditional (or discriminative) model that takes into account information about the input string X.…”

“…To deal with such situations, non-memoryless approaches have been proposed in the literature in the form of probabilistic state machines that are able to take into account the string context. They are mainly based on pair-Hidden Markov Models (pair-HMM) [2,6], probabilistic deterministic automata [1], or stochastic transducers [7]. The string context is described in each state by a statistical distribution over the edit operations.…”

A Spectral Learning Algorithm for Finite State Transducers

“…Most training algorithms for learning FSTs rely on gradient-based or EM optimizations which can be computationally expensive and suffer from local optima issues [8,10]. There are also methods that are based on grammar induction techniques [5,3], which have the advantage of inferring both the structure of the model and the parameters.…”

“…3 . Table 3 shows the time it takes to complete an iteration under EM, together with the number of iterations it takes to reach the best error rates at tests.…”

Melody Recognition with Learned Edit Distances

“…This approach has shown its efficiency in handwritten digit recognition [16] and has been recently extended to tree-structured data [10,14]. Note that non memoryless models have been proposed during the past few years [11,13,15]. While these approaches allow us to model more complex phenomena, their understandability is more complicated because the knowledge is captured in several matrices of finite state machines.…”