Unsupervised induction of stochastic context-free grammars using distributional clustering

Abstract

An algorithm is presented for learning a phrase-structure grammar from tagged text. It clusters sequences of tags together based on local distributional information, and selects clusters that satisfy a novel mutual information criterion. This criterion is shown to be related to the entropy of a random variable associated with the tree structures, and it is demonstrated that it selects linguistically plausible constituents. This is incorporated in a Minimum Description Length algorithm. The evaluation of unsupervised models is discussed, and results are presented when the algorithm has been trained on 12 million words of the British National Corpus.

Missing Generalizations: A Supervised Machine Learning Approach to L2 Written Production

“…Bod 2009 and references therein) and does thus not constrain the computational realization of the statistical induction processes underlying language learning (cf. Clark, 2001;Klein and Manning, 2002;Zuidema, 2006;Bod & Smets, 2012;inter alia). In this paper, we were interested to what extent advanced L2 learners have succeeded in identifying generalizations pertaining to variables that figure in psycholinguistic accounts of sentence-level processing (e.g.…”

Learning Grammar Specifications from IGT: A Case Study of Chintang

“…Grammar induction (Clark, 2001;Klein and Manning, 2002;Klein and Manning, 2004;Haghighi and Klein, 2006;Smith and Eisner, 2006;Snyder et al, 2009, inter alios) involves the learning of grammars from unlabeled sentences. Here, unlabeled means that the sentences are often POS tagged, but no syntactic structures for the sentences are available.…”

SEQUIN: a grammar inference framework for analyzing malicious system behavior

“…They subsequently cluster syntactic units until the grammar has been constructed. For example, EMILE [1] clusters expressions that occur in the same context, while CDC [10] creates sets of sequences within a context before selecting clusters that satisfy the MDL principle (see above).…”

“…The principle finds its primary application in data reduction, where "any regularity in a given set of data can be used to compress the data" [20]. Examples include CDC [10] and e-GRIDS [38]. -Greedy search algorithms make decisions based on their internal logic which may lead to the creation, removal or fusion of rules.…”

“…Overgeneralization occurs when the inference process produces a grammar whose language is larger than the unknown language. The use of negligible items results in an unnecessarily evolutionary GA-based [42] L A g t s [ 8] heuristic ALLiS [13] Inductive CYK [36] ABL [54] MDL e-GRIDS [38] CDC [10] VEGGIE [4,5] Eiland et al [17] greedy search ADIOS CDC Incremental parsing [3,44] Sequitur [37] GraphViz [45,46] clustering EMILE [1] C D C large grammar. To limit the impact of over-generalization, it is recommended to also use a set of negative examples.…”