Six solutions for more reliable infant research

Abstract

Infant research is often underpowered, undermining the robustness and replicability of our findings. Improving the reliability of infant measures offers a solution for increasing statistical power independent of sample size. Here, we discuss two senses of the term reliability in the context of infant research: reliable (large) effects and reliable measures. We examine the circumstances under which effects are strongest and measures are most reliable, and provide simulations to illustrate the relationship between effect size, measurement reliability, and statistical power. We then present six concrete solutions for improving measurement in infant research: (1) routinely estimating and reporting the effect size and measurement reliability of infant tasks, (2) selecting the best measurement tool, (3) developing better infant paradigms, (4) collecting more data points per infant, (5) excluding unreliable data from analysis, and (6) conducting more sophisticated data analyses. Deeper consideration of measurement in infant research will improve our ability to study infant development.

A Global Perspective on Testing Infants Online: Introducing ManyBabies-AtHome

“…The first benefit, participant sample, is two-fold and pertains both to sample size and sample diversity. Many studies in developmental psychology suffer from low statistical power (Bergmann, et al, 2018) due to small sample sizes and limited number of observations per participant (Byers-Heinlein et al, 2021). Online testing allows researchers to test larger samples in less time, because (1) participants can participate in parallel; (2) there is no need to schedule the session; and (3) the study is accessible to participants who cannot come to the lab.…”

Effects of language mixing on bilingual children's word learning

“…Using a similar approach for proportion target looking during the 4 test trials (k = 4), the estimated ICC for test trials was 0.81, 95% CI [0.71, 0.88]. These values reflect good reliability (Koo & Li, 2016), indicating that children showed stable individual variability in our task, which could potentially be accounted for by our predictors of interest (for a discussion of the role of reliability in infant research, see Byers-Heinlein, Bergmann, & Savalei, 2021).…”

A Global Perspective on Testing Infants Online: Introducing ManyBabies-AtHome

“…The first benefit, participant sample, is 2-fold and pertains both to sample size and sample diversity. Many studies in developmental psychology suffer from low statistical power ( Bergmann et al, 2018 ) due to small sample sizes and limited number of observations per participant ( Byers-Heinlein et al, 2021 ). Online testing has the potential to allow researchers to test larger samples in less time, because (1) participants can participate in parallel; (2) there is no need to schedule the session; and (3) the study is accessible to participants who cannot come to the lab.…”