Reducing the Probability of False Positives in Malingering Detection of Social Security Disability Claimants

Abstract

The Symptom Validity Scale (SVS) for low-functioning individuals (Chafetz, Abrahams, & Kohlmaier, 2007) employs embedded indicators within the Social Security Psychological Consultative Examination (PCE) to derive a score validated for malingering against two criterion tests: Test of Memory Malingering (TOMM) and Medical Symptom Validity Test (MSVT). When any symptom validity test is used with Social Security claimants there is a known rate of mislabeling (1-specificity), essentially calling a performance biased (invalid) when it is not, also known as a false-positive error. The great costs of mislabeling an honest claimant necessitated the present study, designed to show how multiple positive findings reduce the potential for mislabeling. This study utilized a known-groups design to address the impact of using multiple embedded indicators within the SVS on the diagnostic probability of malingering. Using four SVS components, Sequence, Ganser, and Coding errors, along with Reliable Digit Span (RDS), the positive predictive power was computed directly or by the chaining of likelihood ratios. The posterior probability of malingering increased from one to two to three failed indicators. With three failed indicators, there were essentially no false positive errors, and the total SVS score was in the range consistent with Definite Malingering, as shown in Chafetz et al. (2007). Thus, in a typical PCE when an examiner might have only a few embedded indicators, more confidence in a diagnosis of malingering might be obtained with a finding of multiple failures.

Comparison of Credible Patients of Very Low Intelligence and Non-Credible Patients on Neurocognitive Performance Validity Indicators

“…Victor and Boone (2007) had cautioned that PVTs validated on individuals of normal intelligence might not be appropriate in populations of extremely low intelligence, and that new measures might need to be developed. Research by Chafetz andcolleagues (2007, 2012) and, more recently, Musso, Barker, Jones, Roid, and Gouvier (2011), illustrate development of PVTs specific for a low IQ population, while the current study shows how scores from existing PVTs, as well as scores from some standard neurocognitive measures, can be modified to be effective in the differential of actual versus feigned low IQ.…”

“…Two recent investigations are noteworthy in this regard, although sensitivity rates were much lower than those achieved in the current study. Chafetz (2011) examined four indicators from the Symptom Validity Scale, and reported that any combination of failures produced high posterior probabilities, with three failures associated with a posterior probability of ≥ 99%, although sensitivities for multiple failures were low (11% to 34%). Shandera and colleagues (2010) observed that when using cut-offs adjusted for low IQ for TOMM trial 2 and retention trial (< 60% correct) and the Digit Memory Test (< 80% correct) in a sample of adults with mental retardation, failure on one or more of the three scores was associated with 92% specificity, and failure on two or more scores resulted in 100% specificity.…”

Examination of Performance Validity Test Failure in Relation to Number of Tests Administered

“…Subsequent research has demonstrated increased specificity with use of multiple PVTs in comparison to single tests. Vickery et al (2004) reported that failure on one of Chafetz (2011) extended the use of multiple PVTs to evaluation of social security disability claimants, which can be a challenging context given concern that lower intellectual ability may contribute to PVT failure (Dean, Victor, Boone, & Arnold, 2008). Chafetz demonstrated that combinations of embedded PVTs resulted in improved classification accuracy over individual use even in a population with potential for an elevated false positive rate.…”

Validity Issues in Atkins Death Cases

“…Even without the full use of the SVS, and even in a posthoc manner, one could review the findings in an Atkins case in which a Wechsler scale was used: If 3þ indicators were positive, that finding would yield considerable evidence for malingering. In particular, if one starts with a base-rate of 30-40% or more (Ardolf, Denney, & Houston, 2007), and 3þ indicators from the SVS were positive, the positive predictive power (PPP) of those findings can be shown to be 99þ% (Chafetz, 2011a). As the formula for PPP is True Positives (TP) 7 (TP þ FP), the findings suggest that the False Positive (FP) rate is minimal (approaching 0).…”

“…This article discusses two methods to deal with this issue. First, failure of multiple PVTs or embedded indicators that are normally passed in well-motivated low IQ individuals leads to a high Positive Predictive Power for malingering (Chafetz, 2011a). Second, regression equations can be utilized to predict the IQ level if performance effort had been optimal (Chafetz et al, 2007).…”