Comparing Automatic and Manual Measures of Parent–Infant Conversational Turns: A Word of Caution

Abstract

The Language ENvironment Analysis system (LENA) records children's language environment and provides an automatic estimate of adult-child conversational turn count (CTC). The present study compares LENA's CTC estimate to manually coded CTC on a sample of 70 English-speaking infants recorded longitudinally at 6, 10, 14, 18, and 24 months of age. At each age, LENA's CTC was significantly higher than manually coded CTC (all ps < .001, Cohen's ds: 0.9-2.05), with the largest discrepancies between the two methods observed at younger ages. The Limits of Agreement Analyses confirm wide disagreements between the two methods, highlighting potential problems with automatic measurement of parent-infant verbal interaction. These findings suggest that future studies should validate LENA's CTC estimates with manual coding. The authors thank Denise Padden and Neva Corrigan for valuable assistance throughout data collection. They also thank

Habla conmigo, daddy! Fathers’ language input in North American bilingual Latinx families

“…The LENA data preparation procedures followed those outlined in previously published studies (Ferjan Ramírez 2018, 2020, 2021; Ramírez‐Esparza et al, 2014, 2016, 2017). Parent and child speech were quantified through a combination of automatic annotation by LENA software and manual (human) annotation.…”

“…This means that an unknown proportion of LENA’s CTCs are identified in error, such as when a parent is talking on the phone and the infant is babbling to herself nearby (i.e., accidental contiguity). The frequency of accidental contiguity has recently been shown to be high in the age range studied here (Ferjan Ramírez et al, 2021), leading us to limit our analysis of turn‐taking to manually identified CTs. In summary, recent validation studies demonstrate that LENA is a useful tool for studying infants’ language environments, but one that should be supplemented by manually quantified measures, as we do in the present study (see Table 1, second column).…”

“…We used FAN and MAN to approximate maternal and paternal word counts, respectively. As in previously published studies (Ferjan Ramírez et al, 2018, 2020, 2021; Ramírez‐Esparza et al, 2014, 2016, 2017) ADEX was used to identify intervals with the language activity of interest (high AWC) for manual analysis, in order to avoid coding when there is no social or linguistic activity (for example, during naps). Each participant’s two daily recordings were segmented into 30‐s intervals.…”

“…To collect a broad range of environments, we required that the selected intervals be spaced at least 3 min apart. Three research assistants, fluent in Spanish and in English, followed the procedures outlined in Ramírez‐Esparza et al (2014, 2016, 2017) and Ferjan Ramírez et al (2018, 2020, 2021). The annotators listened to each 30‐s interval and determined the presence or absence of each variable within the following categories: Speech Partners (mother, father, other adult); Language (English, Spanish, English and Spanish, any other language); Speaking Style (Parentese; Standard Speech); Infant Vocalizations (Babbling).…”

Language and socioemotional development in early childhood: The role of conversational turns

“…In one Spanish‐speaking study (Weisleder & Fernald, 2013), a correlation of 0.80 between LENA and human transcription was obtained. However, one recent study found important discrepancies in the turns measured by LENA and human transcription in the 6–24 month range (Ferjan‐Ramírez et al., 2021).…”