Spectral learning of weighted automata

Abstract

In recent years we have seen the development of efficient provably correct algorithms for learning Weighted Finite Automata (WFA). Most of these algorithms avoid the known hardness results by defining parameters beyond the number of states that can be used to quantify the complexity of learning automata under a particular distribution. One such class of methods are the so-called spectral algorithms that measure learning complexity in terms of the smallest singular value of some Hankel matrix. However, despite their simplicity and wide applicability to real problems, their impact in application domains remains marginal to this date. One of the goals of this paper is to remedy this situation by presenting a derivation of the spectral method for learning WFA that-without sacrificing rigor and mathematical elegance-puts emphasis on providing intuitions on the inner workings of the method and does not assume a strong background in formal algebraic methods. In addition, our algorithm overcomes some of the shortcomings of previous work and is able to learn from statistics of substrings. To illustrate the approach we present experiments on a real application of the method to natural language parsing.

Nonparametric System identification of Stochastic Switched Linear Systems

“…Proposition 1 indicates that H (∞) plays the role of traditional Hankel matrix in LTI systems theory for SLS. Similar subspace based methods for system identification has been discovered in mildly different forms for HMM parameter recovery in [23], [24] or weighted automaton parameter identification in [25].…”

“…Note that in Eq. (25) as N S increase ǫ decreases, i.e., the estimate H (N) becomes better and indeed if ǫ = 0 =⇒ H (N) = H (∞) =⇒ r = n.…”

“…For large enough N S , we have for the choice ofN , r in Eq. (22), (25) respectively that i , B (r) ) are r-order balanced truncated model given in Eq. (16).…”

Proceedings of the SIGFSM Workshop on Statistical NLP and Weighted Automata

“…Then, the spectral learning is applied to the resulting Hankel matrix to obtain WFA. Balle et al (2014) also, offer the main results in spectral learning which are an interesting alternative to the classical EM algorithms in the context of grammatical inference and show the computational efficiency of these algorithms.…”

“…Balle and Mohri (2012) and Balle et al (2014) introduce a new family of algorithms for learning general WFA and stochastic WFA based on the combination of matrix completion problem and spectral methods. These algorithms are designed for learning an arbitrary weighted automaton from sample data of strings and assigned labels.…”