Psychometric Stability of Nationally Normed and Experimental Decoding and Related Measures in Children with Reading Disability

Abstract

Achievement and cognitive tests are used extensively in the diagnosis and educational placement of children with reading disabilities (RD). Moreover, research on scholastic interventions often requires repeat testing and information on practice effects. Little is known, however, about the test—retest and other psychometric properties of many commonly used measures within the beginning reader population, nor are these nationally normed or experimental measures comparatively evaluated. This study examined the test—retest reliability, practice effects, and relations among a number of nationally normed measures of word identification and spelling and experimental measures of achievement and reading-related cognitive processing tests in young children with significant RD. Reliability was adequate for most tests, although lower than might be ideal on a few measures when there was a lengthy test—retest interval or with the reduced behavioral variability that can be seen in groups of beginning readers. Practice effects were minimal. There were strong relations between nationally normed measures of decoding and spelling and their experimental counterparts and with most measures of reading-related cognitive processes. The implications for the use of such tests in treatment studies that focus on beginning readers are discussed.

Evaluating the Efficacy of Remediation for Struggling Readers in High School

“…The WRMT-R reading subtests were administered as pre- and posttest measures, as were the Blending Words subtest of the CTOPP, the Sound Combinations subtest of the Sound-Symbol Test (Lovett et al, 1994; Lovett, Lacerenza, Borden, et al, 2000), and the Challenge Words Test (Lovett et al, 1994; Lovett, Lacerenza, Borden, et al, 2000). The experimental measures have been used in previous intervention studies conducted by our group, have been revealed to be sensitive measures of change in reading-related processes, and have been established to demonstrate sound psychometric properties (Cirino et al, 2002). The Sound Combinations task requires students to provide pronunciations for a set of 30 letter cluster sounds (prompting for alternate pronunciations when appropriate) including vowel digraphs ( ee, oa, ai, igh ), diphthongs ( oo, oy, oi, ou ), and vowel-controlled consonants ( ge, gi, ce, ci ).…”

Prediction and Stability of Mathematics Skill and Difficulty

“…Test–retest reliability (and practice effect information) is certainly available for most broadband standardized instruments, but these data are primarily for a subsample of the norming group (often within the average range) and focused on group-level (rather than individual-level) change. Cirino et al (2002) examined the stability of experimental and standardized measures in a sample of young students with reading difficulty and found few systematic practice effects and adequate test–retest reliability. Even here though, individual-level change was not evaluated, and analogous studies within the math domain are lacking.…”

Dyslexia treatment studies: A systematic review and suggestions on testing treatment efficacy with small effects and small samples

“…Unsurprisingly, two recent randomized controlled trials, reviewed in Study 1, reported the test-retest correlation (Wang, 2017;Wang, Liu, & Xu, 2019), and specified that this measure was .94 according to the test battery, but only .81 and .78, respectively, when calculated in their own samples of children with dyslexia. In addition, Cirino et al (2002) reported that the test-retest correlation among standard scores from major reading batteries ranged between .46 and .92, with a median of about .70, in a sample of 78 children with reading disability. This is below the reliability levels generally reported by standardized batteries (for example, 13 out of 40 studies that we reviewed in our meta-analysis in Study 1 reported test-retest correlations from the normative samples of the standardized batteries that they used, and the range of values was from .71 to .96).…”