Forced-choice pre-employment personality assessment: Construct validity and resistance to faking

O’Neill, Thomas A.; Lewis, Rhys J.; Law, Stephanie; Larson, Nicole; Hancock, Samantha; Radan, Jelena; Lee, Naomi; Carswell, Julie J.

doi:10.1016/j.paid.2016.03.075

Cited by 29 publications

(12 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the most promising methods to diminish the faking problem is the use of the forced-choice (FC) format when answering self-report questions (for other methods, see Rothstein & Goffin, 2006; Wetzel, Böhnke, & Brown, 2016). Some studies showed that FC formats reduced the effects of faking on test scores (e.g., Hirsh & Peterson, 2008), but other studies showed mixed or no effect of the FC format (e.g., Heggestad, Morrison, Reeve, & McCloy, 2006; O’Neill et al, 2016). Indeed, the use of FC formats may have the potential to reduce the faking problem, but as Brown (2016) recently discussed, FC techniques are not likely to solve this problem.…”

Section: Self-reports In High-stakes Assessmentmentioning

confidence: 99%

On the Use of Broadened Admission Criteria in Higher Education

Niessen

Meijer

2017

Perspect Psychol Sci

View full text Add to dashboard Cite

There is an increasing interest in the use of broadened criteria for admission to higher education, often assessed through noncognitive instruments. We argue that there are several reasons why, despite some significant progress, the use of noncognitive predictors to select students is problematic in high-stakes educational selection and why the incremental validity will often be modest, even when studied in low-stakes contexts. Furthermore, we comment on the use of broadened admission criteria in relation to reducing adverse impact of testing on some groups, and we extend the literature by discussing an approach based on behavioral sampling, which showed promising results in Europe. Finally, we provide some suggestions for future research.

show abstract

Section: Self-reports In High-stakes Assessmentmentioning

confidence: 99%

On the Use of Broadened Admission Criteria in Higher Education

Niessen

Meijer

2017

Perspect Psychol Sci

View full text Add to dashboard Cite

show abstract

“…The question regarding the most appropriate response format to gain valid information concerning test takers' characteristics is currently becoming a "hot topic" within the assessment of personality in the job context with reference to the newly emerged forced choice format (see Usami et al, 2016;Lin and Brown, 2017;O'Neill et al, 2017;Brown and Maydeu-Olivares, 2018;Wetzel and Greiff, 2018). However, studies addressing response formats within the domain of the assessment of knowledge, skills, and abilities with stem-equivalent items are rare.…”

Section: Introductionmentioning

confidence: 99%

Effects of Response Format on Psychometric Properties and Fairness of a Matrices Test: Multiple Choice vs. Free Response

2020

View full text Add to dashboard Cite

Reasoning is regarded to be an essential facet of fundamental cognitive abilities. As examinee characteristics may affect performance in Reasoning tests, concern about maintaining fairness is expressed. The purpose of the current study was to examine effects of response format on psychometric properties and fairness of a matrices test according to examinee's sex, risk propensity, and test anxiety. A total of 433 German-speaking pupils (aged 14-20) were randomly assigned to either a multiple choice or a free response version of the same 25-item test. Data analysis yielded Rasch-homogeneous 23-item versions, with higher reliability, but lower criterion validity for the free response test. No interactions between response format and sex, test anxiety, or risk propensity were revealed, but a significant main effect of sex: men out-performed women in reasoning irrespective of response format. Results are discussed with reference to attributes of the test situation and sample characteristics.

show abstract

“…Criterion-related validity coefficients for the 11 different samples for CTT and TIRT scoring methods are presented in Table 1.2 Tests of dependent correlations (|Z|; Steiger, 1980) were used to compare the correlations across scoring methods. Contrary to expectations, all of the comparisons favored CTT over TIRT; two of these were statistically 1 Please note that test-retest reliability is generally considered to be more appropriate for estimating FC reliability (O'Neill, et al, 2017). 2 Note that the operational validity estimates for CTT scoring may be lower than the ones used for commercial purposes by the test vendor because some personality assessment items were eliminated from the present analyses.…”

Section: Resultsmentioning

confidence: 63%

Criterion-related Validity of Forced-Choice Personality Measures: A Cautionary Note Regarding Thurstonian IRT versus Classical Test Theory Scoring

Fisher

Robie

Christiansen

et al. 2019

PAD

View full text Add to dashboard Cite

This study examined criterion-related validity for job-related composites of forced-choice personality scores against job performance using both Thurstonian Item Response Theory (TIRT) and Classical Test Theory (CTT) scoring methods. Correlations were computed across 11 different samples that differed in job or role within a job. A meta-analysis of the correlations (k = 11 and N = 613) found a higher average corrected correlation for CTT (mean ρ = .38) than for TIRT (mean ρ = .00). Implications and directions for future research are discussed.

show abstract

Forced-choice pre-employment personality assessment: Construct validity and resistance to faking

Cited by 29 publications

References 20 publications

On the Use of Broadened Admission Criteria in Higher Education

On the Use of Broadened Admission Criteria in Higher Education

Effects of Response Format on Psychometric Properties and Fairness of a Matrices Test: Multiple Choice vs. Free Response

Criterion-related Validity of Forced-Choice Personality Measures: A Cautionary Note Regarding Thurstonian IRT versus Classical Test Theory Scoring

Contact Info

Product

Resources

About