Short-Term Variability in Cognitive Performance and the Calibration of Longitudinal Change

Abstract

Recent studies have documented that normal adults exhibit considerable variability in cognitive performance from one occasion to another. We investigated this phenomenon in a study in which 143 adults ranging from 18 to 97 years of age performed different versions of 13 cognitive tests in three separate sessions. Substantial within-person variability was apparent across 13 different cognitive variables, and there were also large individual differences in the magnitude of within-person variability. Because people differ in the amount of short-term variability, we propose that this variability might provide a meaningful basis for calibrating change in longitudinal research. Correlations among the measures of within-person variability were very low, even after we adjusted for reliability, and there was little evidence that increased age was associated with a larger amount of within-person variability.

Implications of within-person variability in cognitive and neuropsychological functioning for the interpretation of change.

“…To illustrate, in studies by Salthouse and colleagues, the within-person (across-session) standard deviation averaged about 50% of the between-person standard deviation for a variety of different cognitive variables (e.g., Nesselroade & Salthouse, 2004;Salthouse & Berish, 2005;Salthouse, Nesselroade, & Berish, 2006). This phenomenon of sizable within-person variability is interesting for at least three reasons.…”

“…Some of what is interpreted as change, or lack of change, may therefore be attributable to short-term fluctuation and sampling variation rather than true change. The situation is even more complicated if people vary in the amount of within-person variability, because the same absolute amount of change could have different meanings in different individuals (e.g., Salthouse, Kausler, & Saults, 1986;Salthouse et al, 2006).One possible solution to the concerns about imprecise assessment and ambiguity of change involves determining an individual's short-term fluctuation at each measurement occasion and then using the distribution of scores at each occasion to express that person's change in individually determined t-score units. This proposal is schematically illustrated in Figure 1, with each measurement occasion consisting of three separate assessments.…”

“…A measurement burst study therefore inevitably involves a trade-off between the number of assessments necessary to obtain a reasonable estimate of variability and the pragmatic difficulties associated with extensive testing. The design implemented in a recent project in my laboratory involves administering three different versions of 16 tests in three sessions within a period of about 2 weeks (see, e.g., Salthouse et al, 2006).The cognitive tests have been described in other reports (e.g., Salthouse, Atkinson, & Berish, 2003;Salthouse & Ferrer-Caja, 2003;Salthouse et al, 2006) and are briefly summarized in Table 1. Unlike many previous studies of within-person variability, our studies examined a broad variety of cognitive variables and not just various types of reaction time.…”

“…This is similar to the procedure In the current studies, reliability was estimated by determining separate scores from the oddnumbered and even-numbered items in each test at each session, computing the standard deviations of the scores from the odd-numbered and even-numbered items across the three sessions for each individual, and then, with these two standard deviations as items in coefficient alpha, estimating the reliability of the within-person standard deviation for the entire test. Because 51 individuals had completed the same three-session cognitive test battery between 1 and 2 years earlier, stability coefficients were also computed in Study 3 to provide independent lower-bound estimates of reliability.Although measures of within-person variability are often related to age, Salthouse and colleagues (e.g., Nesselroade & Salthouse, 2004;Salthouse & Berish, 2005;Salthouse et al, 2006) found little unique relation of within-person variability to age after controlling for the individual difference variation in the mean. Because measures of variability may not have any distinct diagnostic value if most of the individual differences in measures of variability are shared with individual differences in the individual's average level of performance, it is important that these results be replicated.…”

“…A measurement burst study therefore inevitably involves a trade-off between the number of assessments necessary to obtain a reasonable estimate of variability and the pragmatic difficulties associated with extensive testing. The design implemented in a recent project in my laboratory involves administering three different versions of 16 tests in three sessions within a period of about 2 weeks (see, e.g., Salthouse et al, 2006).…”