Seeking a Valid Gold Standard for an Innovative, Dialect-Neutral Language Test

Abstract

PURPOSE In this study, the authors explored alternative gold standards to validate an innovative, dialect-neutral language assessment. METHOD Participants were 78 African American children, ages 5;0 (years;months) to 6;11. Twenty participants had previously been identified as having language impairment. The Diagnostic Evaluation of Language Variation-Norm Referenced (DELV-NR; Seymour, Roeper, & J. de Villiers, 2005) was administered, and concurrent language samples (LSs) were collected. Using LS profiles as the gold standard, sensitivity, specificity, and other measures of diagnostic accuracy were compared for diagnoses made from the DELV-NR and participants' clinical status prior to recruitment. In a second analysis, the authors used results from the first analysis to make evidence-based adjustments in the estimates of DELV-NR diagnostic accuracy. RESULTS Accuracy of the DELV-NR relative to LS profiles was greater than that of prior diagnoses, indicating that the DELV-NR was an improvement over preexisting diagnoses for this group. Specificity met conventional standards, but sensitivity was somewhat low. Reanalysis using the positive and negative predictive power of the preexisting diagnosis in a discrepant-resolution procedure revealed that estimates for sensitivity and specificity for the DELV-NR were .85 and .93, respectively. CONCLUSION The authors found that, even after making allowances for the imperfection of available gold standards, clinical decisions made with the DELV-NR achieved high values on conventional measures of diagnostic accuracy.

Classification Accuracy of Teacher Ratings When Screening Nonmainstream English-Speaking Kindergartners for Language Impairment in the Rural South

“…Within the test manual, Seymour et al report that, in a sample of 176 children, a cut score of −1 SD from the normative mean was 94% accurate in classifying children as LI or TD; sensitivity was 95%, and specificity was 93%. Use of the DELV-NR for clinical practice has been supported by two additional studies (Pearson, de Villiers, Magaziner, Perisho, & Sutherland, 2005;Pearson, Jackson, & Wu, 2014). In a previous study with some of the children studied here, DELV-NR syntax scores also correlated with scores on a sentence recall task (Oetting, McDonald, Seidel, & Hegarty, 2016).…”

Sample Length Affects the Reliability of Language Sample Measures in 3-Year-Olds: Evidence From Parent-Elicited Conversational Samples

“…In clinical practice, some attention has been placed on verifying the reliability and validity of standardized tests (e.g., Bogue et al, 2014;Gray, Plante, Vance, & Henrichsen, 1999;Greenslade et al, 2009;Hutchinson, 1996;McCauley & Swisher, 1984;Pearson, Jackson, & Wu, 2014;Perona, Plante, & Vance, 2005;Plante & Vance, 1994Restrepo et al, 2006;Spaulding, Plante, & Farinella, 2006;Ukrainetz & Blomquist, 2002). However, the evidence for reliability and validity of LSA remains relatively sparse (Eisenberg, Fersko, & Lundgren, 2001).…”

Unbiased Language Assessment: Contributions of Linguistic Theory

“…In fact, the final norm-referenced DELV achieved acceptable levels of sensitivity and specificity. Pearson et al (2014) offer a detailed discussion of this topic. These authors compared complex indices of spoken speech analysis as a validity index to complement clinicians' judgment, and achieved a specificity level of 0.91 and a sensitivity level of 0.97.…”