Assessing performance validity during attention-deficit/hyperactivity disorder evaluations: Cross-validation of non-memory embedded validity indicators

Ausloos-Lozano, Jenna E.; Bing-Canar, Hanaan; Khan, Humza; Singh, Palak G; Wisinger, Amanda M; Rauch, Andrew; Buckley, Caitlin M Ogram; Petry, Luke G; Jennette, Kyle J; Soble, Jason R.; Resch, Zachary J

doi:10.1080/87565641.2022.2096889

Cited by 9 publications

(4 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several other tests showed lower sensitivities in the present study than previously estimated. Both versions of the TMT were less sensitive to the instructed simulation of ADHD than the detection of invalid performance described in earlier studies [43,47,61,70] and more akin to the overall lower estimates reported by Ausloos-Lozano and colleagues [69]. The sensitivity of the Stroop Tests' Reading Interference Trial was comparable to results Shura and colleagues [47] noted for a raw-score-based cut-off value for the Word Reading Trial, yet lower than classification accuracies described elsewhere [59][60][61].…”

Section: Discussionsupporting

confidence: 53%

“…The Processing Speed Index (PSI), measured as part of the Wechsler Adult Intelligence Scale-IV (WAIS-IV) [64], has shown variable sensitivity and may put populations with cognitive impairments at risk of false-positive classifications [23,41,65,66]. Its overall acceptable classification accuracy has lent support to the use of the Trail Making Test (TMT) [67] as an adjunct marker of performance validity [23,43,47,61,[68][69][70][71], yet its variations have yielded varying estimates of sensitivity across studies. Complex variations of digit spans (e.g., RDS-WM using the WAIS-IV) [64,72] measuring working memory, detected invalid performance in the assessment of ADHD with adequate accuracy in a study conducted by Bing-Canar and colleagues [73].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Feigning Adult ADHD on a Comprehensive Neuropsychological Test Battery: An Analogue Study

Becke

Tucha

Butzbach

et al. 2023

IJERPH

View full text Add to dashboard Cite

The evaluation of performance validity is an essential part of any neuropsychological evaluation. Validity indicators embedded in routine neuropsychological tests offer a time-efficient option for sampling performance validity throughout the assessment while reducing vulnerability to coaching. By administering a comprehensive neuropsychological test battery to 57 adults with ADHD, 60 neurotypical controls, and 151 instructed simulators, we examined each test’s utility in detecting noncredible performance. Cut-off scores were derived for all available outcome variables. Although all ensured at least 90% specificity in the ADHD Group, sensitivity differed significantly between tests, ranging from 0% to 64.9%. Tests of selective attention, vigilance, and inhibition were most useful in detecting the instructed simulation of adult ADHD, whereas figural fluency and task switching lacked sensitivity. Five or more test variables demonstrating results in the second to fourth percentile were rare among cases of genuine adult ADHD but identified approximately 58% of instructed simulators.

show abstract

Section: Discussionsupporting

confidence: 53%

Section: Introductionmentioning

confidence: 99%

Feigning Adult ADHD on a Comprehensive Neuropsychological Test Battery: An Analogue Study

Becke

Tucha

Butzbach

et al. 2023

IJERPH

View full text Add to dashboard Cite

show abstract

“…Despite the above methodological strengths, some study limitations are noteworthy. Specifically, the memory-based criterion PVTs we used to establish valid and invalid study groups reflect some of the most popular PVTs in clinical practice; however, some examinees may “pick and choose” ability domains in which to underperform, so the degree to which the criterion PVTs appear to assess the same domain may influence their observed accuracy (Ausloos-Lozano et al, 2022). Outside of some support for the FTT (e.g., Arnold et al, 2005; Axelrod et al, 2014), EVIs derived from motor tests generally have received less empirical attention, such that there is a paucity of well-validated motor EVIs that can be used as criterion measures when attempting to cross-validate GPB as an EVI.…”

Section: Discussionmentioning

confidence: 99%

Using the Grooved Pegboard Test as an Embedded Validity Indicator in a Mixed Neuropsychiatric Sample with Varying Cognitive Impairment: Cross-Validation Problems

et al. 2023

Self Cite

View full text Add to dashboard Cite

Embedded validity indicators (EVIs) derived from motor tests have received less empirical attention than those derived from tests of other neuropsychological abilities, particularly memory. Preliminary evidence suggests that the Grooved Pegboard Test (GPB) may function as an EVI, but existing studies were largely conducted using simulators and population samples without cognitive impairment. In this study we aimed to evaluate the GPB’s classification accuracy as an EVI among a mixed clinical neuropsychiatric sample with and without cognitive impairment. This cross-sectional study comprised 223 patients clinically referred for neuropsychological testing. GPB raw and T-scores for both dominant and nondominant hands were examined as EVIs. A known-groups design, based on ≤1 failure on a battery of validated, independent criterion PVTs, showed that GPB performance differed significantly by validity group. Within the valid group, receiver operating characteristic curve analyses revealed that only the dominant hand raw score displayed acceptable classification accuracy for detecting invalid performance (area under curve [AUC] = .72), with an optimal cut-score of ≥106 seconds (33% sensitivity/88% specificity). All other scores had marginally lower classification accuracy (AUCs = .65–.68) for differentiating valid from invalid performers. Therefore, the GPB demonstrated limited utility as an EVI in a clinical sample containing patients with bona fide cognitive impairment.

show abstract

“…Although variants of CPTs vary widely in task characteristics and stimulus material, the accuracy (expressed in errors of omissions and commissions) and speed of responses (expressed in reaction times and variability of reaction times) are the most commonly derived test variables and provide solid predictive accuracy for noncredible cognitive performance [47]. Other cognitive tests that are commonly applied in the routine neuropsychological assessment of adult ADHD, and that also hold promising features for credibility testing in ADHD assessments, include the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) Digit Span (testing short-term and working memory, with a sensitivity of 35-48% and specificity ≥ 85%; [48]), the Trail Making Test (test for processing speed and mental flexibility, with a sensitivity of 29-36% and specificity ≥ 89%; [49]), and verbal fluency (testing divergent thinking, with a sensitivity of 29% and specificity of 91%; [49]). However, the mentioned studies did not examine the utility of a large number of EVIs across multiple cognitive domains on the same sample of participants, which restricts a direct comparison of their value as EVIs.…”

Section: Adhd Research Using Embedded Validity Indicatorsmentioning

confidence: 99%

Cognitive Underperformance in a Mixed Neuropsychiatric Sample at Diagnostic Evaluation of Adult ADHD

Dong,

Koerts,

Pijnenborg

et al. 2023

JCM

View full text Add to dashboard Cite

(1) Background: The clinical assessment of attention-deficit/hyperactivity disorder (ADHD) in adulthood is known to show non-trivial base rates of noncredible performance and requires thorough validity assessment. (2) Objectives: The present study estimated base rates of noncredible performance in clinical evaluations of adult ADHD on one or more of 17 embedded validity indicators (EVIs). This study further examines the effect of the order of test administration on EVI failure rates, the association between cognitive underperformance and symptom overreporting, and the prediction of cognitive underperformance by clinical information. (3) Methods: A mixed neuropsychiatric sample (N = 464, ADHD = 227) completed a comprehensive neuropsychological assessment battery on the Vienna Test System (VTS; CFADHD). Test performance allows the computation of 17 embedded performance validity indicators (PVTs) derived from eight different neuropsychological tests. Further, all participants completed several self- and other-report symptom rating scales assessing depressive symptoms and cognitive functioning. The Conners’ Adult ADHD Rating Scale and the Beck Depression Inventory-II were administered to derive embedded symptom validity measures (SVTs). (4) Results and conclusion: Noncredible performance occurs in a sizeable proportion of about 10% up to 30% of individuals throughout the entire battery. Tests for attention and concentration appear to be the most adequate and sensitive for detecting underperformance. Cognitive underperformance represents a coherent construct and seems dissociable from symptom overreporting. These results emphasize the importance of performing multiple PVTs, at different time points, and promote more accurate calculation of the positive and negative predictive values of a given validity measure for noncredible performance during clinical assessments. Future studies should further examine whether and how the present results stand in other clinical populations, by implementing rigorous reference standards of noncredible performance, characterizing those failing PVT assessments, and differentiating between underlying motivations.

show abstract

Assessing performance validity during attention-deficit/hyperactivity disorder evaluations: Cross-validation of non-memory embedded validity indicators

Cited by 9 publications

References 46 publications

Feigning Adult ADHD on a Comprehensive Neuropsychological Test Battery: An Analogue Study

Feigning Adult ADHD on a Comprehensive Neuropsychological Test Battery: An Analogue Study

Using the Grooved Pegboard Test as an Embedded Validity Indicator in a Mixed Neuropsychiatric Sample with Varying Cognitive Impairment: Cross-Validation Problems

Cognitive Underperformance in a Mixed Neuropsychiatric Sample at Diagnostic Evaluation of Adult ADHD

Contact Info

Product

Resources

About