Using generalized additive modeling to empirically identify thresholds within the ITERS in relation to toddlers' cognitive development.

Abstract

Research linking high-quality child care programs and children’s cognitive development has contributed to the growing popularity of child care quality benchmarking efforts such as quality rating and improvement systems (QRIS). Consequently, there has been an increased interest in and need for approaches to identifying thresholds, or cut-points, in the child care quality measures used in these benchmarking efforts that differentiate between different levels of children’s cognitive functioning. To date, research has provided little guidance to policymakers as to where these thresholds should be set. Using the Early Childhood Longitudinal Study, Birth Cohort (ECLS-B) data set, this study explores the use of generalized additive modeling (GAM) as a method of identifying thresholds on the Infant/Toddler Environment Rating Scale (ITERS) in relation to toddlers’ performance on the mental development subscale of the Bayley Scales of Infant Development (the Bayley Mental Development Scale Short Form–Research Edition, or BMDSF-R). Our findings suggest that simple linear models do not always correctly depict the relationships between ITERS scores and BMDSF-R scores and that GAM-derived thresholds were more effective at differentiating among children’s performance levels on the BMDSF-R. Additionally, our findings suggest that there is a minimum threshold on the ITERS that must be exceeded before significant improvements in children’s cognitive development can be expected. There may also be a ceiling threshold on the ITERS, such that beyond a certain level, only marginal increases in children’s BMDSF-R scores are observed.

Identifying baseline and ceiling thresholds within the Qualistar Early Learning Quality Rating and Improvement System

“…For example, with respect to the thresholds found on the ECERS-R relative to the Cognitive Composite, only five percent of the observations fell below the ECERS-R baseline threshold score of 3.4. Notably, the baseline threshold of 3.4 found on the ECERS-R in this study is similar to the baseline threshold of 3.8 found on a nationally-representative study that included the ITERS (Setodji et al, 2013), which is a parallel measure to the ECERS-R, but used to measure quality within toddler-aged classrooms. The fact that the two similarly constructed measures yielded comparable baseline thresholds provides some suggestive evidence for the replicability of the thresholds identified in this study, but ultimately, research that identifies thresholds with larger samples is needed.…”

“…To identify thresholds, we followed the same approach as Setodji et al (2013), and adopted a GAM model of the form: $${\mathit{\text{Outcome}}}_i=\mu +f_{1}false(X_{1i}false)+\cdots +f_{p}false(X_{\mathit{\text{pi}}}false)+gfalse({\mathit{\text{Quality}}}_{i}false)+{\epsilon }_i,$$, where for child i , Outcome i represents the outcome, Quality i denotes the vector of child care quality, characteristics, X pi (k = 1,2,…,p) indicates the child-level covariates, and ε i is a random error, commonly assumed to be normally distributed with mean 0 and constant variance. In the above equation, f 1 ,…, f p and g represent unknown, non-linear functions that are estimated non-parametrically.…”

“…Because the GAM plots can conceivably support a number of different but equally viable cut-points within a range, analysts using GAM may need to confirm the validity of their chosen thresholds. Following the procedures proposed by Setodjiet al(2013), we validated the GAM-derived thresholds through piecewise regression, which tests whether the slopes between a particular quality measure and outcomes vary across the different regions defined by the thresholds.…”

“…Using data from the Early Childhood Longitudinal Study-Birth Cohort, Setodji, Le, and Schaack (2013) identified thresholds on the 7-point Infant Toddler Environment Rating Scale (ITERS) relative to toddlers’ cognitive development. Departing from piecewise regression techniques in which the analyst determines a priori the thresholds to test, Setodji et al (2013) used generalized additive modeling (GAM) to identify thresholds that were informed by the empirical relationship between the ITERS and the Bayley’s Scale of Mental Development.…”

“…Departing from piecewise regression techniques in which the analyst determines a priori the thresholds to test, Setodji et al (2013) used generalized additive modeling (GAM) to identify thresholds that were informed by the empirical relationship between the ITERS and the Bayley’s Scale of Mental Development. They found that the relationship between ITERS and cognitive development was significantly positive within the ITERS score range of 3.8 and 4.6, but null outside of this range.…”