ALICE: An open-source tool for automatic measurement of phoneme, syllable, and word counts from child-centered daylong recordings

Abstract

Recordings captured by wearable microphones are a standard method for investigating young children's language environments. A key measure to quantify from such data is the amount of speech present in children's home environments. To this end, the LENA recorder and software-a popular system for measuring linguistic input-estimates the number of adult words that children may hear over the course of a recording. However, word count estimation is challenging to do in a language-independent manner; the relationship between observable acoustic patterns and language-specific lexical entities is far from uniform across human languages. In this paper, we ask whether some alternative linguistic units, namely phone(me)s or syllables, could be measured instead of, or in parallel with, words in order to achieve improved cross-linguistic applicability and comparability of an automated system for measuring child language input. We discuss the advantages and disadvantages of measuring different units from theoretical and technical points of view. We also investigate the practical applicability of measuring such units using a novel system called Automatic LInguistic unit Count Estimator (ALICE) together with audio from seven child-centered daylong audio corpora from diverse cultural and linguistic environments. We show that language-independent measurement of phoneme counts is somewhat more accurate than syllables or words, but all three are highly correlated with human annotations on the same data. We share an open-source implementation of ALICE for use by the language research community, enabling automatic phoneme, syllable, and word count estimation from child-centered audio recordings.

Day-long mobile audio recordings reveal multi-timescale dynamics in infant vocal productions and auditory experiences

“…Automated annotation algorithms include the LENA system's proprietary software as well as open-source alternatives including DiViMe (Le Franc et al, 2018), ALICE (Räsänen et al, 2021), and others (e.g., Schuller et al, 2017). LENA annotates recordings with a closed set of mutually exclusive sound source labels and estimates counts of adult words, child vocalizations, and back-and-forth conversational turns between the child and adults.…”

“…Infrastructure supporting sharing data and protocols (e.g., Gilmore et al, 2018;VanDam et al, 2016) helps to maximize value of such investments. For example, sharing manual annotations provides training and evaluation for machine algorithms (e.g., Le Franc et al, 2018;Räsänen et al, 2021;Schuller et al, 2017), which in turn provides new tools for annotating day-long recordings.…”

SCALa: A blueprint for computational models of language acquisition in social context

“…datasets 2020), may provide the ideal starting point for this, since it is cross-cultural and contains images that may allow the identification of nonverbal elements of the social interactions. ACLEW project members have illustrated the importance of coordinated data annotation for developing initial annotations , as well as the usefulness of collaborating with experts of speech technology and machine learning to develop tools that speed up annotation and generalize analyses from the hand-annotated fraction to the day-long scale (Al Futaisi et al, 2019;Le Franc et al, 2018;Räsänen et al, 2020).…”

Using big data from long-form recordings to study development and optimize societal impact

“…The most commonly used algorithm, part of the Language ENvironment Analysis (LENA) system, was trained on data from English-learning infants and young children (up to four years old), but in recent years has been used with a much wider range of ages and languages (see Ganek and Eriks-Brophy 2018 for a review). More recently, open-sourced alternatives to LENA have been developed by members of the ACLEW project (Lavechin, Bousbib, Bredin, Dupoux, & Cristia, 2021;Räsänen et al, 2019;Räsänen, Seshadri, Lavechin, Cristia, & Casillas, 2020), including a system to identify speakers and another to count words, syllables, and phones, all trained LONG-FORM RECORDINGS 7 on multilingual datasets (henceforth, the ACLEW pipeline). Independent assessments comparing automated annotations against human ones suggest that the LENA and ACLEW algorithm accuracy varies widely across participants (even across English-speaking participants: Cristia, Lavechin, et al 2020;Lehet, Arjmandi, Dilley, and Houston 2020;Räsänen et al 2020).…”