Using a supervised machine learning algorithm for detecting faking good in a personality self‐report

Calanna, Pierpaolo; Lauriola, Marco; Saggino, Aristide; Tommasi, Marco; Furlan, Sarah

doi:10.1111/ijsa.12279

Cited by 10 publications

(58 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results of Study 1 were highly encouraging, exceeding the accuracy of previously introduced Likert-based approaches (Calanna et al, 2020;Kuncel & Borneman, 2007).…”

Section: Studymentioning

confidence: 82%

“…We note that none of these studies employed cross-validation, meaning these reported results may in fact have overestimated out-of-sample accuracy. Calanna et al (2020) utilized a method conceptually closest to ours, achieving a cross-validated accuracy of 76% using response patterns indicative of faking on Likert-type scales as predictors in a logistic regression. Our results for both studies appeared to meet and exceed both this and indeed the vast majority of previously published methods of faking detection in terms of accuracy, and to our knowledge empirically demonstrate for the first time how faking may be detected on FC measures.…”

Section: Discussionmentioning

confidence: 99%

“…Building on this, Calanna et al (2020) recently illustrated how machine-learning algorithms can be used to analyze item-level response patterns and classify participants as either honest or faking. Their approach yielded predictive accuracies of around 74 -76% in classifying individuals as faking good for either a fire fighter or teacher role.…”

mentioning

confidence: 99%

“…Despite the increasing popularity of the FC format, to date no study has experimentally tested the possibility of detecting faking on these types of measures. The key contribution of this study is to introduce a new predictive modelling approach to do so, using an empirical index based on responses expected under faking behavior, akin to the methods of Kuncel & Borneman (2007) and Calanna et al (2020). We propose that an applicant engaging in faking behavior will consistently try to provide a response resembling the ideal ranking on each block to make a better impression.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Forced Choice Measures with an Answer Key: A Predictive Modelling Approach for Detecting Faking Behavior

Hughes¹,

Dunlop²,

Holtrop³

et al. 2021

Preprint

View full text Add to dashboard Cite

Forced choice (FC) personality measures are increasingly popular in research and applied contexts. To date however, no method for detecting faking behavior on this format has been both proposed and empirically tested. We introduce a new methodology for faking detection on FC measures, based on the assumption that individuals engaging in faking try to approximate the ideal response on each block of items. Individuals’ responses are scored relative to the ideal using a model for rank-order data not previously applied to FC measures (Generalized Mallows Model). Scores are then used as predictors of faking in a regularized logistic regression. In Study 1, we test our approach using cross-validation, and contrast generic and job-specific ideal responses. Study 2 replicates our methodology on two measures matched and mismatched on item desirability. We achieved between 80 – 92% balanced accuracy in detecting instructed faking, and predicted probabilities of faking correlated with self-reported faking behavior. We discuss how this approach, driven by trying to capture the faking process, differs methodologically and theoretically to existing faking detection paradigms, and measure and context-specific factors impacting accuracy.

show abstract

“…The results of Study 1 were highly encouraging, exceeding the accuracy of previously introduced Likert-based approaches (Calanna et al, 2020;Kuncel & Borneman, 2007).…”

Section: Studymentioning

confidence: 82%

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Forced Choice Measures with an Answer Key: A Predictive Modelling Approach for Detecting Faking Behavior

Hughes¹,

Dunlop²,

Holtrop³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Whereas the idea to investigate response patterns in order to identify faking goes back to Zickar et al (2004), Calanna et al (2020) recently showed that the use of response patterns (i.e., all of a participant's responses; e.g., all answers to all items on a self-report) outperforms the use of scores (e.g., the test score from a self-report) in faking detection. Apparently, there is relevant information in response patterns that is not mirrored by scores (e.g., Kuncel & Bornemann, 2007;Kuncel & Tellegen, 2009).…”

Section: Large Quantities Of Datamentioning

confidence: 99%

Lying on the Dissection Table: Anatomizing Faked Responses

Röhner¹,

Thoss²,

Schütz³

2021

Preprint

View full text Add to dashboard Cite

Research has shown that even experts cannot detect faking above chance, but recent studies have suggested that machine learning may help in this endeavor. However, faking differs between faking conditions, previous efforts have not taken these differences into account, and faking indices have yet to be integrated into such approaches. We reanalyzed seven data sets (N = 1,039) with various faking conditions (high and low scores, different constructs, naïve and informed faking, faking with and without practice, different measures [self-reports vs. implicit association tests; IATs]). We investigated the extent to which and how machine learning classifiers could detect faking under these conditions and compared different input data (response patterns, scores, faking indices) and different classifiers (logistic regression, random forest, XGBoost). We also explored the features that classifiers used for detection. Our results show that machine learning has the potential to detect faking, but detection success varies between conditions from chance levels to 100%. There were differences in detection (e.g., detecting low-score faking was better than detecting high-score faking). For self-reports, response patterns and scores were comparable with regard to faking detection, whereas for IATs, faking indices and response patterns were superior to scores. Logistic regression and random forest worked about equally well and outperformed XGBoost. In most cases, classifiers used more than one feature (faking occurred over different pathways), and the features varied in their relevance. Our research supports the assumption of different faking processes and explains why detecting faking is a complex endeavor.

show abstract

Quality of hire: expanding the multi-level fit employee selection using machine learning

Shet

Nair

2022

IJOA

View full text Add to dashboard Cite

Purpose Organizational psychologists and human resource management (HRM) practitioners often have to select the “right fit” candidate by manually scouting data from various sources including job portals and social media. Given the constant pressure to lower the recruitment costs and the time taken to extend an offer to the right talent, the HR function has to inevitably adopt data analytics and machine learning for employee selection. This paper aims to propose the “Quality of Hire” concept for employee selection using the person-environment (P-E) fit theory and machine learning. Design/methodology/approach The authors demonstrate the aforementioned concept using a clustering algorithm, namely, partition around mediod (PAM). Based on a curated data set published by the IBM, the authors examine the dimensions of different P-E fits and determine how these dimensions can lead to selection of the “right fit” candidate by evaluating the outcome of PAM. Findings The authors propose a multi-level fit model rooted in the P-E theory, which can improve the quality of hire for an organization. Research limitations/implications Theoretically, the authors contribute in the domain of quality of hire using a multi-level fit approach based on the P-E theory. Methodologically, the authors contribute in expanding the HR analytics landscape by implementing PAM algorithm in employee selection. Originality/value The proposed work is expected to present a useful case on the application of machine learning for practitioners in organizational psychology, HRM and data science.

show abstract

Using a supervised machine learning algorithm for detecting faking good in a personality self‐report

Cited by 10 publications

References 37 publications

Forced Choice Measures with an Answer Key: A Predictive Modelling Approach for Detecting Faking Behavior

Forced Choice Measures with an Answer Key: A Predictive Modelling Approach for Detecting Faking Behavior

Lying on the Dissection Table: Anatomizing Faked Responses

Quality of hire: expanding the multi-level fit employee selection using machine learning

Contact Info

Product

Resources

About