Utility of the D–KEFS color word interference test as an embedded measure of performance validity after traumatic brain injury

Donders, Jacobus; Hayden, Ashley

doi:10.1080/13854046.2020.1861659

Cited by 14 publications

(9 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Of the numerous freestanding measures developed over the years, PVTs based on recognition memory have been classically considered best for the detection of noncredible performance ( Bashem et al, 2014 ; Bigler, 2014 ; Inman & Berry, 2002 ; Strauss et al, 2002 ). Recent research on indicators derived from strategies other than recognition has shown that most failed to attain the classification rates of traditional forced-choice indices ( Donders & Hayden, 2020 ; Messa et al, 2020 ), despite optimistic theoretical premises and robustness to genuine severe impairment. A recent meta-analysis on detection strategies confirmed such findings.…”

Section: Types Of Validity Indicatorsmentioning

confidence: 99%

Evaluation of Various Detection Strategies in the Assessment of Noncredible Memory Performance: Results of Two Experimental Studies

et al. 2021

View full text Add to dashboard Cite

Objective: This article investigates the accuracy of individual and combined indicators based on different strategies for detecting noncredible performance as part of a new test for the continuous assessment of short-term memory. Method: In two independent studies, we assessed three groups of simulators, cognitively impaired patients, and nonimpaired community members with four tasks separated by a distractor. Results: Pairwise comparisons between receiver operating characteristic (ROC) curves revealed significant differences between two clusters of indicators: mean recognition, inconsistent responses in recognition, and false positives (area under the ROC curves > .800) proved more accurate than delayed recall and false negatives (area under the ROC curves < .800) in discriminating simulators from patients. Likewise, both studies revealed that adding the false positives indicator based on cued recall to mean recognition incrementally improved classification accuracy (including sensitivity, positive predictive value, and negative predictive value ) compared with the recognition indicator alone. Conclusions: Our results support the association of two distinct indicators for the assessment of noncredible performance, of which one should be a forced-choice indicator.

show abstract

Section: Types Of Validity Indicatorsmentioning

confidence: 99%

Evaluation of Various Detection Strategies in the Assessment of Noncredible Memory Performance: Results of Two Experimental Studies

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The results from the present study also largely found support for the composite indicators developed by Eglit et al (2020) and those developed by Cutler et al (2022) with all indicators clearing 90% specificity utilizing the previously proposed cutoffs except the sum of Conditions 3 and 4. Overall, cutoffs tended to yield lower levels of sensitivity using the previously established cutoff for composite indicators compared to research in TBI (Cutler et al, 2022; Donders & Hayden, 2022; Eglit et al, 2020). In fact, utilizing a more liberal cutoff tended to increase sensitivity.…”

Section: Discussionmentioning

confidence: 81%

“…Moreover, these findings may also be attributed to differing combinations of criterion PVTss within studies given that different combinations of criterion measures may yield varying levels of specificity and sensitivity. Although cutoffs derived from individual CWIT conditions have performed well in mixed clinical and psychoeducational samples, utilizing indicators from individual conditions may lead to an inflation of false-positive rates (Donders & Hayden, 2022).…”

Section: Discussionmentioning

confidence: 99%

“…Interestingly, when Donders and Hayden (2022) attempted to cross-validate all three composite indicators proposed by Eglit et al (2020) utilizing a sample of nonveterans with TBI (uncomplicated mild and moderate severe) most of the indicators failed to reach 90% specificity including the combined ACSS for all four conditions (.80–.92). As a result, Donders and Hayden (2022) suggested that additional research was needed prior to utilizing the proposed EVIs in clinical practice. In response to the lack of consistency across studies, Cutler et al (2022) sought to further investigate the CWIT in a follow-up study in patients with uncomplicated mild TBI and disability benefit applicants.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Utility of the D-KEFS color–word interference test as an embedded validity indicator in psychoeducational evaluations.

Robinson¹,

Miller²,

Herring³

et al. 2023

Psychology & Neuroscience

View full text Add to dashboard Cite

Objective: Previous investigations have yielded divergent results regarding the clinical utility of the Delis–Kaplan Executive Function System (D-KEFS) color–word interference test (CWIT) as an embedded validity indicator. The present study sought to cross-validate previously identified indicators and cutoffs in a sample of adults referred for psychoeducational testing. Method: Archival data from 257 students and community members self- referred for a psychoeducational evaluation at a university clinic in the South were analyzed. Individuals who failed 0 criterion measures were included in the credible group and individuals failing 2 or more criterion measures were included in the noncredible group. Individuals who failed only one of the criterion measures were excluded (n = 53), resulting in a final sample of 204 (credible group n = 164; noncredible group n = 40). Results: Individual conditions yielded high specificity (.88–.91) and moderate sensitivity (.30–.45) at a cutoff of ≤6. Composite indicators yielded high specificity (.94–.99) and low to moderate sensitivity (.20–.40) at previously established cutoffs. Utilizing more liberal cutoffs increased sensitivity (.28–.53) at the predictable cost to specificity (.83–.91). Conclusions: While a cutoff of ≤6 resulted in high specificity for most conditions, composite indicators appear to be more robust. Additional research is needed before recommending any specific cutoff be used in clinical practice with individuals seeking psychoeducational assessment.

show abstract

“…Likelihood ratios (LRs), sensitivity, specificity, and overall correct classification (OCC; the sum of true positives and true negatives divided by sample size) were calculated using standard formulas (Grimes & Schulz, 2005). The minimal acceptable level of specificity is .84 (Larrabee, 2003), although higher values (≥.90) are the emerging norm (Boone, 2013;Donders & Hayden, 2020). BR Fail is the proportion of individuals (%) in a sample who failed a PVT at a given cutoff.…”

Section: Discussionmentioning

confidence: 99%

Multivariate Models of Performance Validity: The Erdodi Index Captures the Dual Nature of Non-Credible Responding (Continuous and Categorical)

Erdődi

2022

Assessment

View full text Add to dashboard Cite

This study was designed to examine the classification accuracy of the Erdodi Index (EI-5), a novel method for aggregating validity indicators that takes into account both the number and extent of performance validity test (PVT) failures. Archival data were collected from a mixed clinical/forensic sample of 452 adults referred for neuropsychological assessment. The classification accuracy of the EI-5 was evaluated against established free-standing PVTs. The EI-5 achieved a good combination of sensitivity (.65) and specificity (.97), correctly classifying 92% of the sample. Its classification accuracy was comparable with that of another free-standing PVT. An indeterminate range between Pass and Fail emerged as a legitimate third outcome of performance validity assessment, indicating that the underlying construct is an inherently continuous variable. Results support the use of the EI model as a practical and psychometrically sound method of aggregating multiple embedded PVTs into a single-number summary of performance validity. Combining free-standing PVTs with the EI-5 resulted in a better separation between credible and non-credible profiles, demonstrating incremental validity. Findings are consistent with recent endorsements of a three-way outcome for PVTs ( Pass, Borderline, and Fail).

show abstract

Utility of the D–KEFS color word interference test as an embedded measure of performance validity after traumatic brain injury

Cited by 14 publications

References 33 publications

Evaluation of Various Detection Strategies in the Assessment of Noncredible Memory Performance: Results of Two Experimental Studies

Evaluation of Various Detection Strategies in the Assessment of Noncredible Memory Performance: Results of Two Experimental Studies

Utility of the D-KEFS color–word interference test as an embedded validity indicator in psychoeducational evaluations.

Multivariate Models of Performance Validity: The Erdodi Index Captures the Dual Nature of Non-Credible Responding (Continuous and Categorical)

Contact Info

Product

Resources

About