Are Established Embedded Performance Validity Test Cut-Offs Generalizable to Patients With Multiple Sclerosis?

Abstract

Objective Data for the use of embedded performance validity tests (ePVTs) with multiple sclerosis (MS) patients are limited. The purpose of the current study was to determine whether ePVTs previously validated in other neurological samples perform similarly in an MS sample. Methods In this retrospective study, the prevalence of below-criterion responding at different cut-off scores was calculated for each ePVT of interest among patients with MS who passed a stand-alone PVT. Results Previously established PVT cut-offs generally demonstrated acceptable specificity when applied to our sample. However, the overall cognitive burden of the sample was limited relative to that observed in prior large-scale MS studies. Conclusion The current study provides initial data regarding the performance of select ePVTs among an MS sample. Results indicate most previously validated cut-offs avoid excessive false positive errors in a predominantly relapsing remitting MS sample. Further validation among MS patients with more advanced disease is warranted.

Examining the Clinical Utility of Selected Memory-Based Embedded Performance Validity Tests in Neuropsychological Assessment of Patients with Multiple Sclerosis

“…Current findings suggested specificities for these variables ranged from 0.80-0.90 and 0.90-0.92, respectively. Although the specificity for C-FCR ≤ 15 was 0.98 as reported by Domen and colleagues [38], these results suggested that 20% (i.e., specificity = 0.80) of the credible group were misidentified as noncredible, suggesting that a more conservative cutoff of ≤14 [37] may yield fewer false positives. Additionally, these results further extend Domen and colleagues' [38] research as they revealed that specificities for C-RH and B-RH-neither of which were discussed by Domen and colleagues [38]-hovered around 0.90 (0.89-0.95).…”

“…Although the specificity for C-FCR ≤ 15 was 0.98 as reported by Domen and colleagues [38], these results suggested that 20% (i.e., specificity = 0.80) of the credible group were misidentified as noncredible, suggesting that a more conservative cutoff of ≤14 [37] may yield fewer false positives. Additionally, these results further extend Domen and colleagues' [38] research as they revealed that specificities for C-RH and B-RH-neither of which were discussed by Domen and colleagues [38]-hovered around 0.90 (0.89-0.95). Current findings support their conclusion that various previously validated cutoff scores may avoid excessive false-positive errors in patients with MS.…”

“…In line with emerging literature on this topic [18,38], the authors strongly recommend that clinicians who perform neuropsychological evaluations with patients with MS include at least one standalone PVT in their battery and consider clinical guidelines in interpreting one or more data suggesting noncredible performance [53]. Performance on PVTs may account for extraordinary amounts of variance in neuropsychological test scores [54][55][56].…”

“…Relatedly, current findings, at least in part, further replicate recent work on embedded PVTs in MS. Domen and colleagues [38] reported specificities for previously validated cutoff scores for C-FCR and B-RD ranged from 0.98-0.99 and 0.88-0.93, respectively. Current findings suggested specificities for these variables ranged from 0.80-0.90 and 0.90-0.92, respectively.…”

“…Unfortunately, these embedded PVTs remain infrequently examined in the context of MS despite their potential for clinical value. To the authors' knowledge, only Domen and colleagues [38] have explored this question, and they reported specificity values ≥0.92 for previously validated cutoffs for C-FCR and B-RD, suggesting that traditional cutoffs may not yield high rates of false-positive errors. Importantly, their findings are limited by their decision to exclude patients performing below expected limits on a standalone PVT and subsequently not reporting classification statistics besides specificity (i.e., sensitivity, classification accuracy).…”