In this paper we discuss algorithms for clustering words into classes from unlabelled text using unsupervised algorithms, based on distributional and morphological information. We show how the use of morphological information can improve the performance on rare words, and that this is robust across a wide range of languages.

No abstract

The question of whether humans represent grammatical knowledge as a binary condition on membership in a set of well-formed sentences, or as a probabilistic property has been the subject of debate among linguists, psychologists, and cognitive scientists for many decades. Acceptability judgments present a serious problem for both classical binary and probabilistic theories of grammaticality. These judgements are gradient in nature, and so cannot be directly accommodated in a binary formal grammar. However, it is also not possible to simply reduce acceptability to probability. The acceptability of a sentence is not the same as the likelihood of its occurrence, which is, in part, determined by factors like sentence length and lexical frequency. In this paper, we present the results of a set of large-scale experiments using crowd-sourced acceptability judgments that demonstrate gradience to be a pervasive feature in acceptability judgments. We then show how one can predict acceptability judgments on the basis of probability by augmenting probabilistic language models with an acceptability measure. This is a function that normalizes probability values to eliminate the confounding factors of length and lexical frequency. We describe a sequence of modeling experiments with unsupervised language models drawn from state-of-the-art machine learning methods in natural language processing. Several of these models achieve very encouraging levels of accuracy in the acceptability prediction task, as measured by the correlation between the acceptability measure scores and mean human acceptability values. We consider the relevance of these results to the debate on the nature of grammatical competence, and we argue that they support the view that linguistic knowledge can be intrinsically probabilistic.

This highly readable but game-changing book shows to what extent the "poverty of the stimulus" argument stems from nothing more than poverty of the imagination. A must read for generative linguists.Ivan Sag, Stanford UniversityFor fifty years, the "poverty of the stimulus" has driven "nativist" linguistics. Clark and Lappin challenge the POS and develop a formal foundation for language learning. This brilliant book should be mandatory reading for anyone who wants to understand the most fundamental question in linguistics. Richard Sproat, Oregon Health and Science UniversityClark and Lappin provide a brilliant and wide-ranging re-examination of one of the most important questions in cognitive science: how much innate structure is required to support language acquisition. A remarkable achievement. Nick Chater, Professor of Behavioural Science, University of WarwickThis comprehensive cutting-edge treatise on linguistic nativism skillfully untangles the human capacity to effortlessly learn languages, from claims that this capacity is specific to language.

An algorithm is presented for learning a phrase-structure grammar from tagged text. It clusters sequences of tags together based on local distributional information, and selects clusters that satisfy a novel mutual information criterion. This criterion is shown to be related to the entropy of a random variable associated with the tree structures, and it is demonstrated that it selects linguistically plausible constituents. This is incorporated in a Minimum Description Length algorithm. The evaluation of unsupervised models is discussed, and results are presented when the algorithm has been trained on 12 million words of the British National Corpus.