Diagnostic accuracy of a universal screening multiple gating procedure: A replication study.

Abstract

The purpose of this diagnostic accuracy study was to evaluate the sensitivity and specificity (among other indicators) of three universal screening approaches, including the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS), a SAEBRS-based teacher nomination tool, and a multiple gating procedure (MGP). Each screening approach was compared to the BASC-2 Behavioral and Emotional Screening System (BESS), which served as a criterion indicator of student social-emotional and behavioral risk. All data were collected in a concurrent fashion. Participants included 704 students (47.7% female) from four elementary schools within the Midwestern United States (21.6% were at risk per the BESS). Findings yielded support for the SAEBRS, with sensitivity = .93 (95% confidence interval [.89-.97]), specificity = .91 (.89-.93), and correct classification = .92. Findings further supported the MGP, which yielded sensitivity = .81 (.74-.87), specificity = .93 (.91-.95), and correct classification = .91. In contrast, the teacher nomination tool yielded questionable levels of diagnostic accuracy (sensitivity = .86 [.80-.91], specificity = .74 [.70-.78], and correct classification = .76). Overall, findings were particularly supportive of SAEBRS diagnostic accuracy, suggesting the MGP might also serve as an acceptable approach to universal screening. Other implications for practice and directions for future research are discussed. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

An item analysis of gender differences on the SRSS‐IE in middle school

“…Practitioners could also consider providing professional development for secondary teachers to better recognize mental health concerns in their students or inclusion of student self‐report at the middle school level (e.g., see Green et al, 2017; Kilgus et al, 2018). Other implications from this study could include training for teachers, matched to student and school needs, to ensure consistency across ratings on the SRSS‐IE.…”

A systematic review of effectiveness and cost-effectiveness of school-based identification of children and young people at risk of, or currently experiencing mental health difficulties

“…Some evidence suggests that overall, universal screening may be the most effective method of identification; however, the rate of false-positive results yielded by this method is high (Auger, 2000; 2004; Husky et al ., 2011), so the expectations of teachers, pupils, and parents would need to be managed accordingly. Some findings indicate that multistage models are more accurate (Scott et al ., 2009; Morey et al ., 2015; Sweeney et al ., 2015); however two studies reported that a single assessment with a universal screening measure is sufficient to accurately identify high-risk individuals, and additional assessments and informants do not improve accuracy (Dowdy et al ., 2016; Kilgus et al ., 2018). Teacher nomination yields a higher number of false negative results than universal screening (Campbell, 2004; Dwyer et al ., 2006; Eklund et al ., 2009; Dowdy et al ., 2013; Cunningham and Suldo, 2014).…”

“…Twelve studies compared identification rates from universal screening and school staff nomination models (Tisher, 1995; Auger, 2000; 2004; Campbell, 2004; Dwyer et al ., 2006; Eklund et al ., 2009; Scott et al ., 2009; Dowdy et al ., 2013; Cunningham and Suldo, 2014; Kieling et al ., 2014; Sweeney et al ., 2015; Kilgus et al ., 2018). In four studies positive identification outcomes were verified by subsequent clinical interview (Auger, 2000; 2004; Scott et al ., 2009; Kieling et al ., 2014; Sweeney et al ., 2015), while remaining studies reported rates of overlap in identification between screening and staff nomination.…”

“…Behavioural and socioemotional problems: Seven studies compared identification rates of students with behavioural and socioemotional problems that used universal screening and nomination models (Tyne and Flynn, 1981; Garland, 1995; Dwyer et al ., 2006; Eklund et al ., 2009; Dowdy et al ., 2013; Kieling et al ., 2014; Kilgus et al ., 2018). Only one study verified positive results of screening with subsequent clinical evaluation (Kieling et al ., 2014); remaining studies reported rates of overlaps between model outcomes.…”

“…They are more likely to nominate students who have more severe difficulties (Garland, 1995), and more ODRs (Dowdy et al ., 2013). However, combining universal screening and nomination did not increase the accuracy of identification compared with universal screening alone (Kilgus et al ., 2018). One study found that a parent-completed universal screener more accurately identified students subsequently diagnosed with internalising disorders, compared with a teacher-completed measure (30–46% and 26–34%, respectively).…”