Language Edit Distance and Maximum Likelihood Parsing of Stochastic Grammars: Faster Algorithms and Connection to Fundamental Graph Problems

Abstract

Given a context free language L(G) over alphabet Σ and a string s ∈ Σ * , the language edit distance problem seeks the minimum number of edits (insertions, deletions and substitutions) required to convert s into a valid member of L(G). The well-known dynamic programming algorithm solves this problem in O(n 3 ) time (ignoring grammar size) where n is the string length [Aho, Peterson 1972, Myers 1985. Despite its numerous applications, to date there exists no algorithm that computes exact or approximate language edit distance problem in true subcubic time.In this paper we give the first such algorithm that computes language edit distance almost optimally. For any arbitrary > 0, our algorithm runs inÕ( n ω poly( ) ) time and returns an estimate within a multiplicative approximation factor of (1 + ) with high probability, where ω is the exponent of ordinary matrix multiplication of n dimensional square matrices. It also computes the edit script. We further solve the local alignment problem; for all substrings of s, we can estimate their language edit distance within (1 ± ) factor inÕ( n ω poly( ) ) time with high probability. Next, we design the very first subcubic (Õ(n ω )) algorithm that given an arbitrary stochastic context free grammar, and a string returns the maximum likelihood parsing of that string. Stochastic context free grammars significantly generalize hidden Markov models; they lie at the foundation of statistical natural language processing, and have found widespread applications in many other fields.To complement our upper bound result, we show that exact computation of maximum likelihood parsing of stochastic grammars or language edit distance with insertion-only edits in true subcubic time will imply a truly subcubic algorithm for all-pairs shortest paths, a long-standing open question. This will result in a breakthrough for a large range of problems in graphs and matrices due to subcubic equivalence. By a known lower bound result [Lee 2002], even the much simpler problem of parsing a context free grammar is as hard as boolean matrix multiplication. Therefore any nontrivial multiplicative approximation algorithms for either of the two problems in time o(n ω ) are unlikely to exist.

Fast Algorithms for Parsing Sequences of Parentheses with Few Errors

“…This leads to an exact algorithm that runs in 2 O(d) · n time. In another work, Saha [11] gave an algorithm for computing a (1 + )-approximation to distance to a fixed context-free grammar. The running time of the algorithm is O(n ω / O(1) ).…”

More consequences of falsifying SETH and the orthogonal vectors conjecture

“…It is conjectured to require n 3−o(1) time, and many problems, especially on graphs, are known to be equivalent to or at least as hard as APSP, see e.g. [48, 60,6,3,49,8,27]. 3 The 3-SUM problem is to decide if a given set of n integers contains three that sum to zero.…”

Truly Sub-cubic Algorithms for Language Edit Distance and RNA-Folding via Fast Bounded-Difference Min-Plus Product

“…Due to known conditional lower bound results for parsing [30,1], LED cannot be approximated within any multiplicative factor in time o(n ω ) (unless cliques can be found faster). Interestingly, if we only ask for insertions as edit operations, Sahe also showed that a truly sub-cubic exact algorithm is unlikely due to a reduction from APSP [41,48]. In contrast, here we show that with insertions and deletions (and possibly substitutions) as edit operations, LED is solvable in truly sub-cubic time.…”

“…It is relatively easy to adapt Valiant's parser to this scored parsing problem, the main difference being that Boolean matrix multiplications are replaced by (min, +)-products. It follows that scored parsing can be solved up to logarithmic factors in the time needed to perform one (min, +)-product (see also [41]). In particular, applying Williams' algorithm for the (min, +)-product [51], one can solve scored parsing in O(n 3 /2 Θ( √ log n) ) time, which is the current best running time for this problem.…”

“…BD grammars appear naturally in relevant applications. Consider for example the wellstudied Language Edit Distance problem (LED) [4,34,30,40,41,1,38]. Here we are given a CFG G and a string σ of terminals.…”