Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate

Kirov, Christo; Cotterell, Ryan

doi:10.1162/tacl_a_00247

Cited by 58 publications

(67 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…11 These findings alone are enough to rule out accounts that posit a regular rule (or regular schema), but what should we put in its place? There is no shortage of options: the literature boasts three broad categories of exemplar model, at least 13 connectionist (neural network) models (see Kirov & Cotterell, 2018 for a summary) and the multiple-rules model of Albright and Hayes (2003). 12 Because exemplar models originate in the cognitive psychology literature on categorization (e.g.…”

Section: Morphologically Inflected Wordsmentioning

confidence: 99%

Against stored abstractions: A radical exemplar model of language acquisition

Ambridge

2019

First Language

135

View full text Add to dashboard Cite

The goal of this article is to make the case for a radical exemplar account of child language acquisition, under which unwitnessed forms are produced and comprehended by on-the-fly analogy across multiple stored exemplars, weighted by their degree of similarity to the target with regard to the task at hand. Across the domains of (1) word meanings, (2) morphologically inflected words, (3) n-grams, (4) sentence-level constructions and (5) phonetics and phonology, accounts based on independently-represented abstractions (whether formal rules or prototype categories) fail for two reasons. First, it is not possible to posit abstractions that delineate possible and impossible form; e.g. that (1) rule in pool tables and data tables, but rule out chairs, (2) rule in the past-tense forms netted and bet but rule out * setted and * jet, (3) rule in the bigram f+t but rule out (probabilistically) v+t, (4) rule in both John feared Bill and John frightened Bill but rule out * John laughed Bill, (5) rule in Speaker A but rule out Speaker B as the person who produced a particular word (e.g. Sa’urday). Second, for each domain, empirical data provide evidence of exemplar storage that cannot be captured by putative abstractions: e.g. speakers prefer and/or show an advantage for (1) exemplar variation even within word-meaning ‘category boundaries’, (2) novel inflected forms that are similar to existing exemplars, (3) n-grams that have occurred frequently in the input, (4) SVO sentences with he as SUBJECT and it as OBJECT and (5) repeated productions of ‘the same’ word that are phonologically similar or, better still, identical. An exemplar account avoids an intractable lumping-or-splitting dilemma facing abstraction-based accounts and provides a unitary explanation of language acquisition across all domains; one that is consistent with models and empirical findings from the computational modelling and neuroimaging literature.

show abstract

Section: Morphologically Inflected Wordsmentioning

confidence: 99%

Against stored abstractions: A radical exemplar model of language acquisition

Ambridge

2019

First Language

135

View full text Add to dashboard Cite

show abstract

“…The results above at least indicate that Japanese verbs can be computed online via some generalizations and those generalizations do depend on the direction of morphological inflection, contrary to the conclusion of previous "wug" tests that Japanese verbs are merely stored in the mental lexicon (de Chene, 1982;Vance, 1987Vance, , 1991Klafehn, 2003Klafehn, , 2013. However, although the MGL is "rule-based", the nature of those generalizations is still an open question to be addressed via the systematic comparison with contemporary analogybased models such as Recurrent Neural Networks (RNN: Kirov and Cotterell, 2018) and Naive Discriminative Learning (NDL: Baayen et al, 2011) couched in Word and Paradigm models of morphology (Stump, 2001;Blevins, 2006).…”

Section: The Past Tense Debatementioning

confidence: 99%

Inverting and Modeling Morphological Inflection

Oseki¹,

Sudo²,

Hiromu³

et al. 2019

Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology

View full text Add to dashboard Cite

show abstract

“…RNNs have thus long attracted researchers interested in language acquisition and processing. Their recent successes in large-scale tasks has rekindled this interest (e.g., Frank et al, 2013;Lau et al, 2017;Kirov and Cotterell, 2018;Mc-Coy et al, 2018;Pater, 2018).…”

Section: Introductionmentioning

confidence: 99%

TabulaNearlyRasa:Probing the Linguistic Knowledge of Character-level Neural Language Models Trained on Unsegmented Text

Hahn

Baroni

2019

Transactions of the Association for Computational Linguistics

View full text Add to dashboard Cite

Recurrent neural networks (RNNs) have reached striking performance in many natural language processing tasks. This has renewed interest in whether these generic sequence processing devices are inducing genuine linguistic knowledge. Nearly all current analytical studies, however, initialize the RNNs with a vocabulary of known words, and feed them tokenized input during training. We present a multi-lingual study of the linguistic knowledge encoded in RNNs trained as character-level language models, on input data with word boundaries removed. These networks face a tougher and more cognitively realistic task, having to discover any useful linguistic unit from scratch based on input statistics. The results show that our "near tabula rasa" RNNs are mostly able to solve morphological, syntactic and semantic tasks that intuitively presuppose word-level knowledge, and indeed they learned, to some extent, to track word boundaries. Our study opens the door to speculations about the necessity of an explicit, rigid word lexicon in language learning and usage. * Work partially done while interning at Facebook AI Research. 2 Our input data, test sets and pre-trained models are available at https://github.com/m-hahn/ tabula-rasa-rnns.3 Single-word utterances are not uncommon in childdirected language, but they are still rather the exception than the rule, and many important words, such as determiners, never occur in isolation (Christiansen et al., 2005).

show abstract

Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate

Cited by 58 publications

References 42 publications

Against stored abstractions: A radical exemplar model of language acquisition

Against stored abstractions: A radical exemplar model of language acquisition

Inverting and Modeling Morphological Inflection

TabulaNearlyRasa:Probing the Linguistic Knowledge of Character-level Neural Language Models Trained on Unsegmented Text

Contact Info

Product

Resources

About