Detecting the significance of changes in performance on the Stroop Color-Word Test, Rey's Verbal Learning Test, and the Letter Digit Substitution Test: The regression-based change approach

Abstract

Serial neuropsychological assessment is often conducted to monitor changes in the cognitive abilities of individuals over time. Because practice effects occur and the reliability of test scores is less than perfect, it is difficult to judge whether varying test results should be attributed to chance trends or to real changes in underlying cognitive abilities. In a large sample of adults (age range, 49-81 years), we evaluated the influence of age, gender, and education on test-retest changes in performance after 3 years on Rey's Verbal Learning Test (VLT), the Stroop Color-Word Test (SCWT), and the Letter Digit Substitution Test (LDST). A new statistical method was applied to assess the significance of changes in test performance (i.e., the regression-based change method). The results showed that test-retest changes differed as a function of age for the VLT Total recall 1-3, VLT Total recall 1-5, VLT Delayed recall, and LDST measures. An age 3 gender interaction was found for the SCWT Interference change score, suggesting that the age-related decline in executive functioning after 3 years was more pronounced for males than for females. A normative change table with appropriate corrections for the relevant independent variables was established. (JINS, 2008, 14, 71-80.)

Establishing normative data for repeated cognitive assessment: A comparison of different statistical methods

“…The main reason for this is that practice effects occur in serial testing situations. Practice effects refer to a variety of factors-such as procedural learning, memory for specific items, and increased comfort with formal testing situations (McCaffrey, Duff, & Westervelt, 2000)that result in systematic improvements in test scores at retesting occasions, even though there was no true change in the latent trait that is measured by the cognitive test (Bartels, Wegrzyn, Wiedl, Ackermann, & Ehrenreich, 2010;Calamia et al, 2012;Van der Elst, Van Breukelen, Van Boxtel, & Jolles, 2008). Practice effects are especially pronounced when the testretest intervals are short (e.g., Theisen, Rapport, Axelrod, & Brines, 1998), but they also occur in studies with test-retest intervals of several years (Rönnlund & Nilsson, 2006;Salthouse, Schroeder, & Ferrer, 2004).…”

“…Practice effects are especially pronounced when the testretest intervals are short (e.g., Theisen, Rapport, Axelrod, & Brines, 1998), but they also occur in studies with test-retest intervals of several years (Rönnlund & Nilsson, 2006;Salthouse, Schroeder, & Ferrer, 2004). In the latter case, the changes in the test scores over time reflect the combined influences of practice effects and true changes in the latent cognitive abilities (Van der Elst et al, 2008). Furthermore, the extent to which practice effects occur is affected by person characteristics such as the age and the educational level of a tested person (Mitrushina & Satz, 1991;Rapport, Brines, Axelrod, & Theisen, 1997;Stuss, Stethem, & Poirier, 1987;Van der Elst et al, 2008).…”

“…In the latter case, the changes in the test scores over time reflect the combined influences of practice effects and true changes in the latent cognitive abilities (Van der Elst et al, 2008). Furthermore, the extent to which practice effects occur is affected by person characteristics such as the age and the educational level of a tested person (Mitrushina & Satz, 1991;Rapport, Brines, Axelrod, & Theisen, 1997;Stuss, Stethem, & Poirier, 1987;Van der Elst et al, 2008).…”

“…Failure to take practice effects into account may invalidate the conclusions that are drawn from a serial cognitive assessment (Calamia et al, 2012;Van der Elst et al, 2008). For example, practice effects may mask the cognitive decline in a patient with early dementia, or practice effects may lead to the incorrect conclusion that a stroke patient has benefitted from a rehabilitation program.…”

“…For example, practice effects may mask the cognitive decline in a patient with early dementia, or practice effects may lead to the incorrect conclusion that a stroke patient has benefitted from a rehabilitation program. Normative data for serial cognitive assessment should thus take the testing history and the demographic characteristics of a patient into account, but it is not clear which statistical method is optimal for achieving this aim (Heaton et al, 2001;Van der Elst et al, 2008).…”