Every second counts: A comparison of four dot counting test scoring procedures for detecting invalid neuropsychological test performance.

Abstract

Although performance validity tests (PVTs) are an integral element of neuropsychological assessment, most PVTs have historically been restricted to the memory domain. The Dot Counting Test (DCT) is a nonmemory PVT shown to reliably identify invalid performance. Although several traditional and abbreviated scoring methods have been derived, no study to date has directly compared the available scoring approaches within a single sample. This cross-sectional study cross-validated 4 different DCT scoring approaches, including the traditional rounded E-score proposed within the manual, an unrounded E-score, and 2 abbreviated scoring procedures based on 4-and 6-card versions (DCT-4 and DCT-6, respectively) in a diverse mixed clinical neuropsychiatric sample (N ϭ 132). Validity groups were established by 5 independent criterion PVTs (102 valid and 30 invalid). Receiver operating characteristic curve analyses yielded significant areas under the curve (AUCs ϭ .84Ϫ.86) for the overall sample, with sensitivities of 50%-67% at Ն89% specificity. The DCT scores had outstanding classification accuracy (AUCs Ն .92; sensitivities ϭ 80%Ϫ83%) in the unimpaired group and excellent classification accuracy in the impaired group (AUCs ϭ .79Ϫ.81; sensitivities ϭ 43%Ϫ60%). Whereas negligible differences emerged between the 4 scoring methods for the cognitively intact group, the DCT-4 showed notably stronger psychometric properties among the overall sample in general and the mild cognitive impairment group in particular. Results corroborate previous findings suggesting that the DCT is a robust PVT, regardless of the employed scoring procedure, and replicate support for the abbreviated DCT-4 as the recommended validity indicator. Public Significance StatementThis study provided further cross-validation support for the clinical utility of the Dot Counting Test (DCT) as a brief, nonmemory performance validity test among a diverse neuropsychiatric population and replicated recent novel findings regarding abbreviated DCT scoring approaches that allow for more accurate assessment of validity status while minimizing evaluation time burden and associated costs.

Cross-validating the Dot Counting Test Among an Adult ADHD Clinical Sample and Analyzing the Effect of ADHD Subtype and Comorbid Psychopathology

“…This may have been due to differences in sample characteristics, as this study included patients formally diagnosed with ADHD, rather than those simply referred for ADHD evaluation, as was done in the Marshall et al (2016) study. An additional strength of this study was the comprehensive approach toward examining both rounded and unrounded E-scores similar to previous cross-validations of the DCT (Bailey et al, 2021; McCaul et al, 2018; Rhoads, Resch, et al, 2021). This aspect of this study led to the conclusion that both rounded and unrounded E-scores produce similarly robust psychometric properties when evaluating patients with ADHD.…”

“…In addition, the DCT has brief administration and scoring procedures for a standalone PVT that does not significantly extend battery length. Moreover, it has been cross-validated in neuropsychiatric (McCaul et al, 2018; Rhoads, Resch, et al, 2021), veteran (Bailey et al, 2021; Soble et al, 2018), and Spanish-speaking (Burton et al, 2012; Robles et al, 2015) populations, as well as in populations outside North America (Vilar-López et al, 2008; Weiss & Rosenfeld, 2010).…”

Do demographic factors influence detection of invalid neuropsychological test performance using common performance validity tests? A multisite investigation.

“…These five PVTs were intentionally selected to optimize validity assessment across diverse cognitive domains (i.e., memory vs. nonmemory), as well as using some of the most commonly administered PVTs by practicing neuropsychologists in civilian and VA settings to increase generalizability (Martin et al, 2015; Schroeder et al, 2016; Young et al, 2016). Additionally, all have been extensively cross-validated, and their effectiveness as PVTs has been empirically documented, even among clinical populations with cognitive impairment (e.g., Bain et al, 2021; Cerny, Resch, et al, 2022; Kraemer et al, 2020; Martin et al, 2020; Rhoads, Resch, et al, 2021; Webber & Soble, 2018). Moreover, our decision to use five PVTs is consistent with the median number of PVTs administered throughout a clinical evaluation, per recent practice surveys (Martin et al, 2015).…”

Standalone Performance Validity Tests May Be Differentially Related to Measures of Working Memory, Processing Speed, and Verbal Memory in Patients With Multiple Sclerosis

“…The Dot Counting Test (DCT; Boone et al, 2002) is a standalone PVT with good construct validity in diverse clinical samples (Rhoads, Resch, et al, 2021; Soble et al, 2018). The E-score (DCT) was the variable of interest.…”