Consistently inconsistent: An examination of the variability in the identification of emotional disturbance

Abstract

The federal definition of emotional disturbance (ED) has been heavily criticized as vaguely defined and poorly operationalized yet there has not been a formal analysis of the reliability of the ED criteria. This study examined the reliability of the federal criteria for a special education designation of ED. A total of 179 school psychologists reviewed a mock special education report, made an eligibility determination, and provided information about their eligibility decision. In all, 56 participants found the student met ED eligibility criteria using 16 different combinations of the five criteria. Krippendorff's α, a statistic preferred for content analysis, was calculated as a measure of criterion reliability. Results indicated extremely poor reliability (α = 0.2011). These findings demonstrate the importance of redefining the existing criteria.

Definitions of and Evaluation Procedures for Emotional Disturbance: A Tale of 50 States

“…It goes without saying that eligibility decisions with this type of mindset are easily influenced by biases. As an illustration, Scardamalia et al (2019) asked 179 school psychologists to use the federal ED definition criteria to make an eligibility determination with a case study and found extremely poor reliability. Policy leaders may need to consider how the ambiguity of the social maladjustment exclusion clause can contribute to patterns of differential access to special education based on where a student lives (intra- and interstate variability).…”

Enhancing outcomes: Culturally adapted social–emotional and behavioral interventions for rural black elementary learners at risk.

“…Unchanged since the passage of PL 94-142, experts describe the emotional disturbance definition language as vague and contradictory among a variety of other pejoratives (Sallese et al, 2024). Past research shows extremely poor reliability among school psychologists applying the definition for eligibility decisions (Scardamalia et al, 2019). Vague language within a disability category definition leads to subjective decision making, which is easily influenced by biases (Hanchon & Allen, 2018).…”

Expelled Students in Need of Special Education Services Using Bayes’ Theorem: Implications for the Social Maladjustment Clause?

“…The under-identification of students with emotional behavioral disorders may be due to a variety of other reasons beyond perceived stigma or the occurrence of diagnostic substitution. First, diagnostic criteria for emotional disturbance can be overly broad and ambiguous leading to the inconsistent application of criteria in diagnosis (Scardamalia et al, 2019), which may improve with regular, universal screening but widespread screening still does not occur (Bruhn et al, 2014). In addition, internalizing behaviors such as anxiety and depression can go unnoticed (Mitchell et al, 2019), whereas externalizing behaviors tend to manifest in ways that interfere with classroom instruction.…”