Density and Distinctiveness in Early Word Learning: Evidence From Neural Network Simulations

Abstract

High phonological neighborhood density has been associated with both advantages and disadvantages in early word learning. High density may support the formation and fine-tuning of new word sound memories-a process termed lexical configuration (e.g., Storkel, 2004). However, new high-density words are also more likely to be misunderstood as instances of known words, and may therefore fail to trigger the learning process (e.g., Swingley & Aslin, 2007). To examine these apparently contradictory effects, we trained an autoencoder neural network on 587,954 word tokens (5,497 types, including mono-and multisyllabic words of all grammatical classes) spoken by 279 caregivers to English-speaking children aged 18-24 months. We then simulated a communicative development inventory administration and compared network performance to that of 2,292 children aged 18-24 months. We argue that autoencoder performance illustrates concurrent density advantages and disadvantages, in contrast to prior behavioral and computational literature treating such effects independently. Low network error rates signal a configuration advantage for high-density words, while high network error rates signal a triggering advantage for low-density words. This interpretation is consistent with the application of autoencoders in academic research and industry, for simultaneous feature extraction (i.e., configuration) and anomaly detection (i.e., triggering). Autoencoder simulation therefore illustrates how apparently contradictory density and distinctiveness effects can emerge from a common learning mechanism.

Infant Phonetic Learning as Perceptual Space Learning: A Crosslinguistic Evaluation of Computational Models

“…We find that the model can correctly predict the crosslinguistic pattern for the Mandarin Chinese contrast but not for the Catalan contrast. In Study 2, we consider four other models developed in the speech technology community; these models are all state-of-the-art extensions of the well-known autoencoder neural network (Kramer, 1991) commonly used in modeling statistical language learning (e.g., Jones & Brandt, 2020;Mareschal & French, 2017;Plaut & Vande Velde, 2017). We evaluate these four algorithms on the same three datasets, to study whether any of the algorithms can correctly predict the discrimination patterns for all the three contrasts, potentially providing a better model of infant phonetic learning than the one proposed in Schatz et al (2021).…”

Rational inattention: A new theory of neurodivergent information seeking

“…We modelled neurotypical and neurodivergent information seeking using autoencoder neural networks (Figure 1). Autoencoders have been used to simulate a wide range of child behaviours, from categorisation and visual object processing to curiosity‐driven learning and language acquisition (Jones & Brandt, 2020; Mareschal et al., 2000; Twomey & Westermann, 2018; Westermann et al., 2009; Westermann & Mareschal, 2004, 2012). Autoencoders are a class of self‐supervised neural network, which form representations of their learning environment by adaptively updating internal weights to minimise the difference between the input they receive and a re‐construction of that input that they produce.…”

Rational inattention: A new theory of neurodivergent information seeking

“…We modelled neurotypical and neurodivergent information seeking using autoencoder neural networks (Figure 1). Autoencoders have been used to simulate a wide range of child behaviours, from categorisation and visual object processing to curiosity-driven learning and language acquisition (Jones & Brandt, 2020;Mareschal et al, 2000;Twomey & Westermann, 2018;Westermann et al, 2009;Westermann & Mareschal, 2004, 2012. Autoencoders are a class of self-supervised neural network, which form representations of their learning environment by adaptively updating internal weights to minimise the difference between the input they receive and a re-construction of…”