Screening systems and decision making at the preschool level: Application of a comprehensive validity framework

Abstract

Universal screening is designed to be an efficient method for identifying preschool students with mental health problems, but prior to use, screening systems must be evaluated to determine their appropriateness within a specific setting. In this article, an evidence‐based validity framework is applied to four screening systems for identifying preschool students with mental health problems. The framework is influenced by the most recent standards for educational and psychological testing, research on test accessibility, and considerations for evaluating screening systems. Suggestions are provided for evaluating the accessibility (Step 1), reliability (Step 2), construct validity (Step 3), and consequential validity (Step 4) of an instrument. Other factors for consideration (i.e., developmental stage, incremental validity, and generalizability) are also identified. Special attention is given to conditional probability indices, which are highly relevant to evaluation of screening systems, given the dichotomous nature of decision making in preschool mental health. The authors suggest that this framework be used, along with specification of the construct of interest and characteristics of the environment, to identify the appropriate method to be used for each preschool screening decision. © 2011 Wiley Periodicals, Inc.

Social‐emotional problems among three‐year‐olds differ based on the child's gender and custody arrangement

“…The results are derived from a comprehensive population‐based study with high participation rates. Out of several questionnaires ASQ:SE fulfilled the criteria of systematic identification of children's developmental and behavioural problems with adequate psychometric properties. In addition, support for the use of ASQ:SE was established through the positive experience of CHC staff and parents when using the ASQ:SE as a tool in dialogue.…”

Comparison of Predictive Validity and Diagnostic Accuracy of Screening Measures of Reading Skills

“…There are no widely accepted criteria for conditional probability indices. According to Kettler and Feeney‐Kettler (), index values between .00 and .40 are considered very low, values between .40 and .60 are low, .values of .60 to .80 are moderate, and values of .80 to 1.00 are high. Some researchers have recommended .90 for sensitivity and .80 for specificity (Compton et al., ; Johnson et al., ).…”

Student and Teacher Ratings of Academic Competence: An Examination of Cross-Informant Agreement

“…1 Also in Study 2 (N = 66), conditional probability indices 2 were calculated to characterize agreement between the dichotomous competence scores across the two forms of the ACES. The magnitudes of conditional probability indices were classified using Kettler and Feeney- Kettler's (2011) guidelines. 3 Phi coefficients also were calculated as an indicator of the shared variance between the dichotomized scores from the ACES.…”