The use of unreliable measures constitutes a threat to our understanding of psychopathology, because advancement of science using both behavioral and biologically-oriented measures can only be certain if such measurements are reliable. Two pillars of NIMH’s portfolio – the Research Domain Criteria (RDoC) initiative for psychopathology and the target engagement initiative in clinical trials – cannot succeed without measures that possess the high reliability necessary for tests involving mediation and selection based on individual differences. We focus on the historical lack of reliability of attentional bias measures as an illustration of how reliability can pose a threat to our understanding. Our own data replicate previous findings of poor reliability for traditionally-used scores, which suggests a serious problem with the ability to test theories regarding attentional bias. This lack of reliability may also suggest problems with the assumption (in both theory and the formula for the scores) that attentional bias is consistent and stable across time. In contrast, measures accounting for attention as a dynamic process in time show good reliability in our data. The field is sorely in need of research reporting findings and reliability for attentional bias scores using multiple methods, including those focusing on dynamic processes over time. We urge researchers to test and report reliability of all measures, considering findings of low reliability not just as a nuisance but as an opportunity to modify and improve upon the underlying theory. Full assessment of reliability of measures will maximize the possibility that RDoC (and psychological science more generally) will succeed.

We examined whether normal aging spares or compromises cue-driven spontaneous retrieval processes that support prospective remembering. In Experiment 1, young and older adults performed prospective-memory tasks that required either strategic monitoring processes for retrieval (nonfocal) or for which participants relied on spontaneous retrieval processes (focal). We found age differences for nonfocal, but not focal, prospective-memory performance. Experiments 2 and 3 used an intention-interference paradigm in which participants were asked to perform a prospective-memory task (e.g., press "Q" when the word money appears) in the context of an image-rating task and were then told to suspend their prospective-memory intention until after completing an intervening lexical-decision task. During the lexical-decision task, we presented the exact prospective-memory cue (e.g., money; Experiments 2 and 3) or a semantically related lure (e.g., wallet; Experiment 3), and we inferred spontaneous retrieval from slowed lexical-decision responses to these items relative to matched control items. Young and older adults showed significant slowing when the exact prospective-memory cue was presented. Only young adults, however, showed significant slowing to the semantically related lure items. Collectively, these results partially support the multiprocess theory prediction that aging spares spontaneous retrieval processes. Spontaneous retrieval processes may become less sensitive with aging, such that older adults are less likely to respond to cues that do not exactly match their encoded targets.