The Student Risk Screening Scale: A Reliability and Validity Generalization Meta-Analysis

Abstract

The purpose of this study was to conduct reliability and validity generalization meta-analyses of evidence regarding the Student Risk Screening Scale (SRSS), a universal screener for externalizing behavior problems. A systematic review of the literature resulted in the identification of 17 studies inclusive of evidence regarding SRSS score (a) internal consistency reliability (i.e., alpha coefficients), and/or (b) criterion-related validity (e.g., correlations between the SRSS and various outcomes). Multilevel meta-analyses indicated that across studies, SRSS scores were associated with adequate internal consistency (α = .83). Analyses further suggested the SRSS was a valid indicator of both social and behavioral outcomes (r = .52) and academic outcomes (r = .42). Follow-up analyses suggested that in accordance with theory-driven expectations, the SRSS was a stronger indicator of externalizing problems and broad behavior outcomes relative to alternative outcomes (e.g., internalizing problems). Limitations and directions for future research are discussed, including recommendations for the collection of additional SRSS diagnostic accuracy evidence.

SSIS Performance Screening Guide as an Indicator of Behavior and Academics: A Meta-Analysis

“…Meta-analyses provide stronger, more realistic estimates of the average observed validity coefficients across studies and are important to the identification of trends that support the psychometric defensibility of the analyzed screeners. Similar meta-analyses have been conducted for other emotional and behavioral screeners (Allen et al, 2019;Kilgus et al, 2017).…”

“…These categories were based, in part, on those specified by Kilgus et al (2017). We adapted them based on the specific variables in the articles included in our analysis.…”

Measurement invariance of the Student Risk Screening Scale across time and gender.

“…Despite its strengths, research on the psychometric properties of the SRSS is limited in important ways. First, as Kilgus et al (2017) noted, almost all research on psychometric properties of the SRSS has been conducted by a single research group. Of the 17 studies aggregated in their meta-analysis, 15 were conducted by Lane and colleagues, with disproportionate sampling from a single region of the United States.…”

“…Of the 17 studies aggregated in their meta-analysis, 15 were conducted by Lane and colleagues, with disproportionate sampling from a single region of the United States. Even with the valuable contributions of this research group, more studies examining the psychometric properties of the SRSS conducted by independent research teams in new regions of the United States are important in addressing conceivable researcher and sampling biases (Kilgus et al, 2017). Second, existing investigations of the SRSS have focused especially on its internal consistency, test-retest reliability, and criterion-related validity.…”

“…This study contributes to a large number of studies on the psychometric properties of the SRSS in at least two important ways. First, we are an independent research group who collected data at three schools in the Midwestern region of the United States, responding to calls for new research groups to investigate the psychometric properties of the SRSS in new regions of the United States (Kilgus et al, 2017). Second, this study was undertaken to contribute evidence for the construct validity of the SRSS.…”