SD-IRT models benefit from the measurement rigor of item response theory-which permits the modeling of item difficulty and examinee ability-and from signal detection theory-which provides an interpretive framework encompassing the experimentally validated constructs of memory discrimination and response bias. We used this approach to validate the construct representation of commonly used research measures and to demonstrate how nonoptimized item parameters can lead to erroneous conclusions when interpreting neuropsychological test data. Future work might include the development of computerized adaptive tests and integration with mixture and random-effects models.

Background Bipolar disorder (BD) is associated with inhibitory deficits characterized by a reduced ability to control inappropriate actions or thoughts. While aspects of inhibition such as exaggerated novelty-seeking and perseveration are quantified in rodent exploration of novel environments, similar models are rarely applied in humans. The human Behavioral Pattern Monitor (hBPM), a cross-species exploratory paradigm, has identified a pattern of impaired inhibitory function in manic BD participants, but this phenotype has not been examined across different BD phases. The objective of this study was to determine if euthymic BD individuals demonstrate inhibitory deficits in the hBPM, supporting disinhibition as an endophenotype for the disorder. Methods 25 euthymic BD outpatients and 51 healthy comparison subjects were assessed in the hBPM, where activity was recorded by a concealed videocamera and an ambulatory monitoring sensor. Results Euthymic BD individuals, similar to manic subjects, demonstrated increased motor activity, greater interaction with novel objects, and more frequent perseverative behavior relative to comparison participants. The quantity of locomotion was also reduced in BD individuals treated with mood stabilizers compared to other patients. Limitations Low sample size for treatment subgroups limits the evaluation of specific medication regimens. Conclusions Our results suggest that BD is distinguished by both trait- and state-dependent inhibitory deficits optimally assessed with sophisticated multivariate measures. These data support the use of the hBPM as a tool to elucidate the effects of BD across various illness states, facilitate the development of BD animal models, and advance our understanding of the neurobiology underlying the disorder.

The neurocognitive processes involved during classic spatial working memory (SWM) assessment were investigated by examining naturally preferred eye movement strategies. Cognitively healthy adult volunteers were tested in a computerized version of the Corsi Block-Tapping Task — a spatial span task requiring the short term maintenance of a series of locations presented in a specific order — coupled with eye-tracking. Modeling analysis was developed to characterize eye-tracking patterns across all task phases, including encoding, retention, and recall. Results revealed a natural preference for local gaze maintenance during both encoding and retention, with fewer than 40% fixated targets. These findings contrasted with the stimulus retracing pattern expected during recall as a result of task demands, with 80% fixated targets. Along with participants’ self-reported strategies of mentally “making shapes”, these results suggest the involvement of covert attention shifts and higher-order cognitive Gestalt processes during spatial span tasks, challenging instrument validity as single measure of SWM storage capacity.

Cognitive tasks that are too hard or too easy produce imprecise measurements of ability, which, in turn, attenuates group differences and can lead to inaccurate conclusions in clinical research. We aimed to illustrate this problem using a popular experimental measure of working memory—the N-back task—and to suggest corrective strategies for measuring working memory and other cognitive deficits in schizophrenia. Samples of undergraduates (n = 42), community controls (n = 25), outpatients with schizophrenia (n = 33), and inpatients with schizophrenia (n = 17) completed the N-back. Predictors of task difficulty—including load, number of word syllables, and presentation time—were experimentally manipulated. Using a methodology that combined techniques from signal detection theory and item response theory, we examined predictors of difficulty and precision on the N-back task. Load and item type were the two strongest predictors of difficulty. Measurement precision was associated with ability, and ability varied by group; as a result, patients were measured more precisely than controls. Although difficulty was well matched to the ability levels of impaired examinees, most task conditions were too easy for non-impaired participants. In a simulation study, N-back tasks primarily consisting of 1- and 2-back load conditions were unreliable, and attenuated effect size (Cohen’s d) by as much as 50%. The results suggest that N-back tasks, as commonly designed, may underestimate patients’ cognitive deficits due to non-optimized measurement properties. Overall, this cautionary study provides a template for identifying and correcting measurement problems in clinical studies of abnormal cognition.