When Time is of the Essence: Preliminary Findings for a Quick Administration of the Dot Counting Test

Abstract

Objective Performance validity research has emphasized the need for briefer measures and, more recently, abbreviated versions of established free-standing tests to minimize neuropsychological evaluation costs/time burden. This study examined the accuracy of multiple abbreviated versions of the Dot Counting Test (“quick” DCT) for detecting invalid performance in isolation and in combination with the Test of Memory Malingering Trial 1 (TOMMT1). Method Data from a mixed clinical sample of 107 veterans (80 valid/27 invalid per independent validity measures and structured criteria) were included in this cross-sectional study; 47% of valid participants were cognitively impaired. Sensitivities/specificities of various 6- and 4-card DCT combinations were calculated and compared to the full, 12-card DCT. Combined models with the most accurate 6- and 4-card combinations and TOMMT1 were then examined. Results Receiver operator characteristic curve analyses were significant for all 6- and 4-card DCT combinations with areas under the curve of .868–.897. The best 6-card combination (cards, 1-3-5-8-11-12) had 56% sensitivity/90% specificity (E-score cut-off, ≥14.5), and the best 4-card combination (cards, 3-4-8-11) had 63% sensitivity/94% specificity (cut-off, ≥16.75). The full DCT had 70% sensitivity/90% specificity (cut-off, ≥16.00). Logistic regression revealed 95% classification accuracy when 6-card or 4-card “quick” combinations were combined with TOMMT1, with the DCT combinations and TOMMT1 both emerging as significant predictors. Conclusions Abbreviated DCT versions utilizing 6- and 4-card combinations yielded comparable sensitivity/specificity as the full DCT. When these “quick” DCT combinations were further combined with an abbreviated memory-based performance validity test (i.e., TOMMT1), overall classification accuracy for identifying invalid performance was 95%.

Every second counts: A comparison of four dot counting test scoring procedures for detecting invalid neuropsychological test performance.

“…The variability in optimal cutoff scores across studies may be partially because the current sample appears to have a higher proportion of patients with MCI relative to the previous studies' samples, which tends to result in higher recommended cutoff scores to maintain adequate specificity. This point is further supported by the observation that the cutoff scores recommended by Bailey et al (2019) for their abbreviated DCT-4 and DCT-6 scoring methods were slightly higher than in the current study, which may have been due to differences in study eligibility criteria. Specifically, 17% of the patients with cognitive impairment in the study by Bailey and colleagues met criteria for major neurocognitive disorder, whereas patients with cognitive impairment of that severity were excluded in the present study.…”

“…score of Ն15.25 (63% sensitivity/89% specificity) and DCT-6 cutoff of Ն14.83 (67% sensitivity/90% specificity) for the overall sample were relatively similar to the psychometric properties reported in the original Bailey et al (2019) validation study.…”

Comparing the Independent and Aggregated Accuracy of Trial 1 and the First 10 TOMM Items for Detecting Invalid Neuropsychological Test Performance Across Civilian and Veteran Clinical Samples

“…Two popular options for improving the efficiency of performance validity assessment are the use of (a) embedded PVTs (i.e., deriving a validity index from an already-planned ability test within a neurocognitive test battery) and (b) abbreviated versions of freestanding PVTs. Thus, there has been a recent proliferation of research designed to cross-validate embedded PVTs (e.g., Bailey et al, 2018; Pliskin et al, 2020; Resch, Pham, 2020; Soble, Sharp et al, 2021; Webber & Soble, 2018; White et al, 2020) and abbreviated PVTs (e.g., Bailey et al, 2021; Rhoads, Resch, 2021) in various clinical populations.…”

“…Performance validity tests (PVTs) refer to a diverse collection of measures that have been specifically validated to objectively identify invalid test performance during neuropsychological evaluations (Soble, Webber & Bailey, 2021). Practice standards in clinical neuropsychology call for routine administration of multiple freestanding and embedded PVTs within all neuropsychological evaluations, as this allows for continuous sampling of respondent test engagement to ensure that test scores are an accurate and credible reflection of an examinee's neurocognitive abilities (Boone, 2013;Bush et al, 2005;Sherman et al, 2020;Sweet et al, 2021).…”

A Known-Groups Validation of the Medical Symptom Validity Test and Analysis of the Genuine Memory Impairment Profile

“…In light of this situation, there has been an emerging emphasis on developing briefer versions of traditional freestanding PVTs (e.g., Bailey et al, 2019; Denning, 2012). The Medical Symptom Validity Test (MSVT; Green, 2004) was developed as a shorter analogue to the Word Memory Test (WMT; Green, 2003), with fewer word pairs (10 vs. 20) and a shorter delay interval (10 vs. 30 minutes).…”