Acquiring language from speech by learning to remember and predict

Abstract

Classical accounts of child language learning invoke memory limits as a pressure to discover sparse, language-like representations of speech, while more recent proposals stress the importance of prediction for language learning. In this study, we propose a broadcoverage unsupervised neural network model to test memory and prediction as sources of signal by which children might acquire language directly from the perceptual stream. Our model embodies several likely properties of real-time human cognition: it is strictly incremental, it encodes speech into hierarchically organized labeled segments, it allows interactive top-down and bottom-up information flow, it attempts to model its own sequence of latent representations, and its objective function only recruits local signals that are plausibly supported by human working memory capacity. We show that much phonemic structure is learnable from unlabeled speech on the basis of these local signals. We further show that remembering the past and predicting the future both contribute to the linguistic content of acquired representations, and that these contributions are at least partially complementary.

Character-based PCFG Induction for Modeling the Syntactic Acquisition of Morphologically Rich Languages

“…Future work could explore the extent to which syntactic knowledge can be acquired from lower-level (e.g. phonemic or acoustic) input alone by including a word segmentation task (Elsner and Shain, 2017;Shain and Elsner, 2020) for the model. Additionally, recent work in unsupervised grammar induction (Jin and Schuler, 2020;Zhang et al, 2021) has shown that incorporating visual information in the form of images and videos helps learn constituents that denote entities or action.…”

Introducing meta-analysis in the evaluation of computational models of infant language development

“…In terms of input, some models operate on linguistic abstractions of speech, such as phonemic, phonetic or orthographic transcripts (e.g., Frank et al, 2010;Goldwater et al, 2009;Nikolaus and Fourtassi, 2021), phonetic or lexical representations derived using pre-trained automatic speech recognition systems (e.g., Fourtassi and Dupoux, 2014;Roy, 2005;Salvi et al, 2012), or by using some simplified representations of acoustic speech, such as formant frequencies of pre-segmented vowels (Coen, 2006;de Boer and Kuhl, 2003). Another set of models operate directly on real continuous speech (e.g., Kamper et al, 2016;Nixon, 2020;Park and Glass, 2008;Schatz et al, 2021;Shain and Elsner, 2020). Besides processing language input only, there are models that use visual concurrent input in addition to spoken language (e.g., Alishahi et al, 2017;Chrupa la et al, 2017;Coen, 2006;Harwath et al, 2019;Harwath et al, 2016;Khorrami and Räsänen, 2021;Nikolaus and Fourtassi, 2021;Roy, 2005).…”

“…Instead of analyzing clustering purity, the ABX-test analyzes phonemic discriminability of internal representations learned by a model. In studies focusing on speech segmentation, such as phone (Michel et al, 2016;Räsänen, 2014;Scharenborg et al, 2007), syllable (Räsänen et al, 2018), or word segmentation (Shain and Elsner, 2020), the model is typically a system that has a mechanism to identify temporal positions of unit boundaries in time. These boundaries are then compared to unit boundaries in ground-truth phonetic or word-level annotations.…”

Towards unsupervised phone and word segmentation using self-supervised vector-quantized neural networks

“…Methods for automatically learning phone-or word-like units from unlabelled speech audio could enable speech technology in severely low-resourced settings [1,2] and could lead to new cognitive models of human language acquisition [3][4][5]. The goal in unsupervised representation learning of phone units is to learn features which capture phonetic contrasts while being invariant to properties like the speaker or channel.…”

“…We evaluate these on four different tasks: unsupervised phone segmentation [23], ABX phone discrimination [24], same-different word discrimination [25], and as inputs to a symbolic word segmentation algorithm [1]. The last-mentioned is particularly important since the segmentation and clustering of word-like units remains a major but important challenge [5,26].…”