Predicting Age of Acquisition for Children's Early Vocabulary in Five Languages Using Language Model Surprisal

Abstract

What makes a word easy to learn? Early‐learned words are frequent and tend to name concrete referents. But words typically do not occur in isolation. Some words are predictable from their contexts; others are less so. Here, we investigate whether predictability relates to when children start producing different words (age of acquisition; AoA). We operationalized predictability in terms of a word's surprisal in child‐directed speech, computed using n‐gram and long‐short‐term‐memory (LSTM) language models. Predictability derived from LSTMs was generally a better predictor than predictability derived from n‐gram models. Across five languages, average surprisal was positively correlated with the AoA of predicates and function words but not nouns. Controlling for concreteness and word frequency, more predictable predicates and function words were learned earlier. Differences in predictability between languages were associated with cross‐linguistic differences in AoA: the same word (when it was a predicate) was produced earlier in languages where the word was more predictable.

Mmi01 at The BabyLM Challenge: Linguistically Motivated Curriculum Learning for Pretraining in Low-Resource Settings

“…Furthermore, as a fascinating aspect of cognitive modelling, we assess our models' capability to predict the age of word acquisition (AoA). Based on the work of Portelance et al (2023), computing this metric 2 The code for curriculum formation and training can be found on Github: https://github.com/mi-m1/BabyLM-Entry. involves an estimation of the average surprisal of words in child-directed utterances sourced from CHILDES.…”

Annotating Columns with Pre-trained Language Models

“…Recently, researchers have begun advocating for the use of PLMs for modeling cognitive development in children (Kosoy et al, 2023;Salewski et al, 2024). For example, Portelance et al (2023) and Bhardwaj et al (2024) suggest the use of language models to predict the age of acquisition of words in children. Researchers have also proposed studying second language acquisition and bilingualism by mapping pre-training steps in PLMs to understand the rate of language development (Evanson et al, 2023;Marian, 2023;Sharma et al, 2024).…”

Learning the Meanings of Function Words From Grounded Language Using a Visual Question Answering Model

“…With respect to our third research question, frequency or word predictability is a known predictor of the order in which children acquire words (Braginsky, Yurovsky, Marchman, & Frank, 2019; Goodman, Dale, & Li, 2008; Kuperman, Stadthagen‐Gonzalez, & Brysbaert, 2012; Portelance, Duan, Frank, & Lupyan, 2023). There may, however, be other factors—in relation to or independent from—frequency that makes learning the meaning of certain function words harder than others.…”