Ordinal Factor Analysis of Graded-Preference Questionnaire Data

Abstract

We introduce a new comparative response format, suitable for assessing personality and similar constructs. In this "graded-block" format, items measuring different constructs are first organized in blocks of 2 or more; then, pairs are formed from items within blocks. The pairs are presented one at a time, to enable respondents expressing the extent of preference for one item or the other using several graded categories. We model such data using confirmatory factor analysis (CFA) for ordinal outcomes. We derive Fisher information matrices for the graded pairs, and supply R code to enable computation of standard errors of trait scores. An empirical example illustrates the approach in low-stakes personality assessments and shows that similar results are obtained when using graded blocks of size 3 and a standard Likert format. However, graded-block designs may be superior when insufficient differentiation between items is expected (due to acquiescence, halo or social desirability).

Effects of Applicant Faking on Forced-Choice and Likert Scores

“…We might argue for at least two plausible explanations for our failure to find support for the benefit of the forced-choice format to reduce the effects of applicant faking. First, the forced-choice trait scores in our study had lower reliabilities than those of the corresponding Likert trait scores, given the forced-choice scales provided less information about the measured traits than polytomous rating scales consisting of the same statements (Brown & Maydeu-Olivares, 2011b, 2013, 2017). Consequently, parameter estimates in the regression models for the forced-choice format may have been more severely attenuated than those for the Likert format.…”

Wetzel, Frick, & Brown (2020)_Does multidimensional forced-choice prevent faking?

“…A tutorial and Excel macro for creating the Mplus syntax can be found on http://annabrown.name/software. 2016; Zhang et al, 2019), while generally having lower reliabilities given the same items as their single-stimulus counterparts (Brown & Maydeu-Olivares, 2018b).…”

Though Forced, Still Valid: Psychometric Equivalence of Forced-Choice and Single-Statement Measures

“…They are instructed to choose the statement that is "the most like me" (i.e., the PICK format), or in addition to choose the one that is "the most like me", to also choose the statement that is "the least like me" (i.e., the MOLE format), or to rank all statements within a block (i.e., the RANK format; Brown, 2016a). Some variations to the FC format include the compositional preference task where respondents are required to distribute a fixed number of total points among several statements in a block according to the degree to which these statements describe themselves (Brown, 2016b;Chan, 2003), and the graded preference FC where respondents are asked to indicate how much they prefer statement A to B using a number of ordered categories (Brown & Maydeu-Olivares, 2017).…”

“…They are instructed to choose the statement that is “more like me,” to choose one statement that is “more like me” and one that is “the least like me,” or to rank all statements within a block (Brown, 2016a). Recently, some other variations of the FC format are also proposed (Brown, 2016b; Brown & Maydeu-Olivares, 2017). Statements within a block are usually balanced on social desirability.…”