Effects of Applicant Faking on Forced-Choice and Likert Scores

Abstract

We introduce a novel, regression-based moderation framework to model faking effects that incorporates evaluation of faking tendency as a moderator. We also consider how perceived trait desirability may be factored into the framework and provide programming code for applied researchers to utilize the method in their research. Using this framework, we revisit a well-known response format (i.e., forced-choice) to formally evaluate its ability to mitigate the effects of applicant faking as compared to the widely used Likert format. The impetus for the latter evaluation stems from the use of item response theory (IRT) modeling to yield non-ipsative scores from forced-choice measures. We found strong support for the need to incorporate moderating effects of faking tendency and desirability in predicting applicants’ responses. Also, we found that the only substantial difference across formats lies in forced-choice scores yielding a lower mean inflation at high faking values. As a result, forced-choice scores do not outperform Likert scores when selection ratios are used but may be beneficial when cutoff scores are used. Application of the moderation framework presented extends to self-reported construct measures of varied kinds.

Though Forced, Still Valid: Psychometric Equivalence of Forced-Choice and Single-Statement Measures

“…SS measurement is, however, not without issues. Faking and rating scale errors, such as halo, acquiescent responding, extreme response style, and midpoint response style, lead to serious concerns (e.g., Donovan, Dwight, & Schneider, 2014; Landy & Farr, 1980; Pavlov, Maydeu-Olivares, & Fairchild, 2018). An alternative that was originally developed to overcome problems with SS measures was the forced-choice (FC) scale (Rundquist, 1946).…”

Does multidimensional forced-choice prevent faking? Comparing the susceptibility of the multidimensional forced-choice format and the rating scale format to faking.

“…However, in natural settings, applicants' tendency to fake can play an important role. Pavlov, Maydeu-Olivares, and Fairchild (2019) showed using a regression-based moderation analysis that faking was only reduced for the forced-choice format at high faking tendency levels, though they did not match the forced-choice items regarding desirability. This implies that future research conducted in natural settings should also take participants' propensity for faking into account.…”

The Fakeability of Personality Measurement with Graded Paired Comparisons

“…In addition to meta-analyses, which consider the fakeability of one response format (Likert vs. FC) in isolation, individual studies that have drawn a direct comparison can also be used to assess the relative susceptibility to faking. In fact, the vast majority of these studies found faking-induced score inflation to be smaller for FC scales (Christiansen et al, 2005;Huber, 2017;Jackson et al, 2000;Lee et al, 2019;Pavlov et al, 2019;Vasilopoulos et al, 2006); but, see Heggestad et al (2006) for contrary results.…”

“…It is particularly important for potential faking in personnel selection, since faking in this context would be closely linked to changes in applicants' rank order and, therefore, is directly relevant to selection decisions. While one study, showed that FC scores from an applicant condition explained more "true" trait variance than their Likert scale counterparts (Christiansen et al, 2005), other studies have failed to consistently show that discrete FC scales are superior (Guan, 2015;Heggestad et al, 2006;Pavlov et al, 2019). This might be a consequence of reliability issues associated with discrete FC measures (Guan, 2015;Pavlov et al, 2019) or for the study by Heggestad and colleagues (2006), an artifact of ipsative scoring methods.…”

“…While one study, showed that FC scores from an applicant condition explained more "true" trait variance than their Likert scale counterparts (Christiansen et al, 2005), other studies have failed to consistently show that discrete FC scales are superior (Guan, 2015;Heggestad et al, 2006;Pavlov et al, 2019). This might be a consequence of reliability issues associated with discrete FC measures (Guan, 2015;Pavlov et al, 2019) or for the study by Heggestad and colleagues (2006), an artifact of ipsative scoring methods. Importantly, reliability issues could be counteracted by the GPC format, and ipsative scoring artifacts could be avoided by scoring responses with Thurstonian IRT models as this would facilitate normative trait score estimates.…”