Convergent, Criterion and Social Validity of the Emotional and Behavioral Screener

Abstract

Recent reports suggest that up to 20 % of all school-aged students demonstrate characteristics related to emotional disturbance. However, \1 % of these students actually receive special education services for ED, a percentage that has remained unchanged for nearly 30 years. Screening procedures have been developed to identify the behavioral needs of students within school-wide systems such as Multitiered System of Support, Positive Behavior Support, and Response to Intervention. While several screening tools are available at this time, many require significant time, training, and/or effort, factors which limit their practical use in schools. A new efficient assessment tool Emotional and Behavioral Screener (EBS) has recently been developed as a universal screening instrument. The purpose of this study was to examine the validity of the EBS scores, with a sample of over 600 elementary students, in comparison to scores from the Social Skills Improvement System assessment tool. Results indicated high levels of convergent validity between the scores (social skills; q = -0.70; and problem behavior; q = 0.79); strong criterion validity as assessed by differences in EBS scores between individuals with normal versus problematic scores (based on established cut-off scores) on the SSIS (social skills; d = 2.23; and problem behavior; d = 3.58) and assessed by binary classification between cut-off scores of the EBS and SSIS (AUROC range .86-.88); and adequate social validity. Implications and limitations of these results are discussed.

Latent Structure of Scores From the Emotional and Behavioral Screener

“…Thus, it could be that there was not enough residual variance left to be explained by one of the group factors. Nonetheless, findings from this study, combined with existing research (e.g., Cullinan & Epstein, 2013a; Lambert, Epstein, & Cullinan, 2014; Pierce et al, 2016) support the use scores from the EBS to identify students who are at risk of EBD.…”

“…In addition, research supports the interrater reliability (r = .63), test-retest reliability (r = .90), social validity, and the convergent validity (r = .79, .87) of scores from the EBS (Nordness, Epstein, Cullinan, & Pierce, 2014;Pierce, Lambert, & Alamer, 2016). EBS scores also demonstrate diagnostic utility in accurately classifying students with risk of EBD across age (5-17; 86%-95% classification accuracy) and gender (87% classification accuracy for females; 86%-88% classification for males) groups (Lambert, Epstein, & Cullinan, 2014;Pierce et al, 2016).…”

“…A student who’s Total EBS Score exceeds the cutoff score for their age and gender is identified as “at risk.” The cutoff score for younger male students (5-11) is 5, younger female students (5-11) is 3, older male students (12-17) is 7, and older female students (12-17) is 4. The psychometric properties and diagnostic utility of the EBS scores are well established (Cullinan & Epstein, 2013a, 2013b; Lambert, Epstein, & Cullinan, 2014; Lambert, Epstein, Ingram, et al, 2014; Nordness et al, 2014; Pierce et al, 2016).…”

Differential item functioning across race and ethnicity for the Emotional and Behavioral Screener.

“…EBS scores also demonstrate adequate interrater reliability ( r = .63) and test–retest reliability ( r = .90). Studies of validity also support the convergent validity ( r s = .87 [BASC-2 BESS], .79 [SSiS Problem Behavior]) and social validity of the EBS (Nordness, Epstein, Cullinan, & Pierce, 2014; Pierce, Lambert, & Alamer, 2016). EBS scores also demonstrate diagnostic utility in accurately classifying students with risk of EBD across age (5 to 17; 86% to 95% AUROC) and gender groups (87% AUROC for females; 86–88% AUROC for males; Lambert, Epstein, & Cullinan, 2014).…”

“…The EBS has 10 highly discriminating items that take teachers 1-2 min total per student to complete. Research supports the content validity of the EBS (Cullinan & Epstein, 2013a, 2013b, and there is evidence that scores from the EBS are technically adequate (Lambert, Epstein, & Cullinan, 2014;Nordness, Epstein, Cullinan, & Pierce, 2014;Pierce, Lambert, & Alamer, 2016), can discriminate between students with and without emotional and behavioral difficulties (Pierce et al, 2016), and reflect the theoretical factor structure across age and gender groups (Lambert, Epstein, Ingram, Simpson, & Bernstein, 2014;Lambert, January, & Pierce, 2016). The existing EBS research is promising in its utility for universal screening (see Lambert, Epstein, & Cullinan, 2014); however, research has not yet investigated whether scores on the EBS might differ as a function of a student's race or ethnicity.…”