Sample Size, Statistical Power, and False Conclusions in Infant Looking‐Time Research

Abstract

Infant research is hard. It is difficult, expensive, and time consuming to identify, recruit and test infants. As a result, ours is a field of small sample sizes. Many studies using infant looking time as a measure have samples of 8 to 12 infants per cell, and studies with more than 24 infants per cell are uncommon. This paper examines the effect of such sample sizes on statistical power and the conclusions drawn from infant looking time research. An examination of the state of the current literature suggests that most published looking time studies have low power, which leads in the long run to an increase in both false positive and false negative results. Three data sets with large samples (>30 infants) were used to simulate experiments with smaller sample sizes; 1000 random subsamples of 8, 12, 16, 20, and 24 infants from the overall samples were selected, making it possible to examine the systematic effect of sample size on the results. This approach revealed that despite clear results with the original large samples, the results with smaller subsamples were highly variable, yielding both false positive and false negative outcomes. Finally, a number of emerging possible solutions are discussed.

Promoting Replicability in Developmental Research Through Meta‐analyses: Insights From Language Acquisition Research

“…Consider a series of studies with 80% power (a number typically deemed sufficient), where every fifth result will be a false negative, that means it will not reflect that there is a true effect present in the population. This observation was recently demonstrated by Oakes () by using data from a high‐powered looking time study.…”

Longitudinal development of attention and inhibitory control during the first year of life

“…Not only did the association replicate, it was also the strongest path in the model (β = −.331). This is a particular issue in infant research, where sample sizes tend to be small(Oakes, 2017). Our model controlled for all other associations between variables when estimating each path; therefore, this result indicates that performance on two very different IC tasks was moderately associated at the end of the first year, even when other factors were taken into account and despite the general noisiness of infant data.…”

Opportunities for Neurodevelopmental Plasticity From Infancy Through Early Adulthood

“…Our methods also need to be sensitive to the infant's ability (or willingness) to tolerate our measures. Finally, we will need to step beyond traditional approaches that have limited infant research to small samples with limited measures per child (see Oakes, , for a critique).…”