Detecting change: A comparison of three neuropsychological methods, using normal and clinical samples

Abstract

Detecting change in individual patients is an important goal of neuropsychological testing. However, limited information is available about test-retest changes, and well-validated prediction methods are lacking. Using a large nonclinical subject group (N = 384), we recently investigated test-retest reliabilities and practice effects on the Wechsler Adult Intelligence Scale and Halstead-Reitan Battery. Data from this group also were used to develop models for predicting follow-up test scores and establish confidence intervals around them. In this article we review those findings, examine their generalizability to new nonclinical and clinical groups, and explore the sensitivity of the prediction models to real change. Despite similarities across samples in reliability coefficients and practice effects, limits to the generalizability of prediction methods were found. Also, when multiple test measures were considered together, one or more "significant" changes were common in all (including stable) subject groups. By employing normative cut-offs that correct for this, sensitivity of the models to neurological recovery and deterioration was modest to good. More complex regression models were not more accurate than the simpler Reliable Change Index with correction for practice effects when confidence intervals for all methods were adjusted for variations in level of baseline test performance.

The cognitive effects of chemotherapy in post-menopausal breast cancer patients: a controlled longitudinal study

“…A recent review paper [52] reported that a comparative study in neurologically stable patients found that regression models were better able to predict outcomes than the use of the RCI and produced narrower confidence intervals [53][54][55][56]. Vardy et al recommended either a regression model or the RCI to analyze longitudinal data in this area of study.…”

The (mis)classification of chemo-fog – methodological inconsistencies in the investigation of cognitive impairment after chemotherapy

“…In fact it is likely that the variance of change scores in the patient group is larger than in the control group due to different individual responses to chemotherapy. We feel justified in using reliable change indices with correction for practice (see Heaton et al for a comparison of RCI and other methods of detecting change [15]). While we feel justified in the method we have selected the fact remains that those employed by other researchers produce very different results.…”

Measuring cognitive change in older adults. Do reliable change indices of the SIDAM predict dementia?

“…However, when the prediction of dementia is the only question to be answered (which it is usually not) and when it is at the same time of no interest, whether the patient has reliably changed in cognition, then the computation of RCI does not seem essential. This does not exclude the possibility that RCI are useful for other clinical questions than dementia diagnosis (see e. g. [7]). …”

“…Two earlier studies only investigated an RCI that corrected for practice effects [7,14]. Generally, it seems important that the absolute level of test performance is accounted for (see also [7]). If the RCI+regression is computed, the cognitive level is automatically accounted for.…”

“…Thus, the utility of RCI for diagnostic decisions and especially their utility for dementia diagnosis needs to be proved. It has been suggested that change score methods may be useful for the detection or rather for the prediction of diagnostic change among persons with declining cognitive abilities [5,7,14]. However, there are discrepant findings, whether RCI methods contribute to the prediction of dementia diagnosis in older adults or not [5,7,14].…”