Infancy researchers need to change the way they develop their measures: A comment on Byers‐Heinlein et al. (2021)

Abstract

In their article, Byers-Heinlein et al. (2021) propose six solutions to create more reliable measures for infancy research.Whereas some of their solutions focus on changing the culture in the field (reporting on reliabilities, choosing measures based on psychometrics instead of conventions or face valid-

An examination of measures of young children's interest in natural object categories

“…In other words, even with high measurement reliability, we cannot be certain that the measure indexes the desired effect of interest; in fact, by trying to maximise reliability of measures, we may end up undermining the validity of the measurement (Zettersten et al, 2022). Some of the solutions presented to improve the reliability and validity of infant research are to increase the number of trials, exclude low-quality data prior to analysis, utilise more sophisticated analyses (Byers-Heinlein et al, 2021), developing measurements collaboratively (Reinelt et al, 2022), using a greater variety of exemplars as stimuli (Visser et al, 2022;Zettersten et al, 2022) and using multiple outcome measures to measure the variable of interest (Havron, 2022;LoBue et al, 2020). However, there are issues with many of these solutions: for example, Zettersten and colleagues (2022) argue that with too many trials, one may actually measure children's willingness to perform the task rather than the actual construct of interest.…”

Early cognitive abilities as predictors of later skills and competencies in the context of children's socioeconomic background

“…Regarding infant research, this measurement error should be larger than in other samples, due to processes of maturation and physiological changes that affect the observable behavior (Byers-Heinlein et al, 2021). Potential sources of errors are content sampling (e.g., a poor calibration of the stimulus material), time sampling, and administrative errors (Reynolds et al, 2021a), all of which are prevalent in infant research (Reinelt et al, 2022). Typically, test-retest reliability as an index of stability over time is assessed in studies using habituation-dishabituation tasks .…”

“…While newer studies have implemented more elaborate statistical approaches for investigating children's habituation such as polynomial modeling (e.g., Dahlin, 2004;Lavoie & Desrochers, 2002), such model-based approaches have shortcomings regarding infant-control designs and the measurement of dishabituation (Kucharský et al, 2022). Thus, quantitative index measures are still prevalent in infant research, although there is a recent ongoing discussion about how to approach habituation and dishabituation from a methodological standpoint (Byers-Heinlein et al, 2021;Fassbender, 2022;Kucharský et al, 2022;Reinelt et al, 2022). In contrast to quantitative measures, qualitative measures of habituation (e.g., Bornstein & Benasich, 1986;Mayes & Kessen, 1989) that describe looking time patterns are seldom studied (Kavšek, 2004a).…”

“…It is reasoned that it is theoretically not clear what the measures drawn from such tasks indicate as they blend various aspects of visual attention, learning abilities, memory capacity, and behavioral self-regulation. This perspective often includes the realization that habituation patterns might not be the same for all children (Byers-Heinlein et al, 2021;Fassbender, 2022;Reinelt et al, 2022) and that socioeconomic background may have a considerable impact on how children develop early cognitive abilities (Noble et al, 2015). This is mainly because, from a theoretical perspective, the Stimulus-Model Comparator Theory is used as a general framework Sicard-Cras et al, 2022), although empirical evidence highlights several shortcomings of the model in explaining the reaction phenomena in connection to basic cognitive abilities (Colombo et al, 2010).…”

“…A central strength of the studies was the large and heterogeneous sample that allowed for uncovering that children's socioeconomic background has a substantial effect on children's looking behavior in such tasks. While findings on the predictive validity of the domain-general and domainspecific tasks were partly counterintuitive, the results suggest that habituation patterns might not be the same for all children (Byers-Heinlein et al, 2021;Fassbender, 2022;Reinelt et al, 2022) and that socioeconomic background may have a considerable impact on how children develop early cognitive abilities (Noble et al, 2015). Thus, the large-scale data contributes to the present research in a way that raises fundamental questions regarding the social selectivity and generalizability of findings of small-scale laboratory experiments as well as regarding children's prolonged looking at sequentially presented stimuli, which might reflect aspects of sustained attention and self-regulation.…”