The Journey from Likert to Forced-Choice Questionnaires: Evidence of the Invariance of Item Parameters

Morillo, Daniel; Abad, Francisco; Leenen, Iwin; Hontangas, Pedro M.; Ponsoda, Vicente

doi:10.5093/jwop2019a11

Cited by 31 publications

(28 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The absolute values for the loadings in the FCQ were systematically lower than the ones for the GSQ (0.37 vs. 0.56). These results are consistent with the findings of Morillo et al (2019) on the invariance of discrimination parameters under the GRM and the MUPP-2PL models.…”

Section: Resultssupporting

confidence: 92%

“…The items in the pool have been previously applied to a 5-point Likert scale format to a total of 1531 Psychology undergraduate students at Universidad Autónoma de Madrid, using an incomplete sampling design, and were calibrated under Samejima’s Graded Response Model (GRM; Samejima, 1968). Partial reports on the calibration study can be found in Nieto et al (2017) and Morillo et al (2019). In the calibration study conducted by Nieto et al (2017), the scores based on this item pool showed good convergent validities with the NEO Five-Factor Inventory-3 (McCrae and Costa, 2007), which is a brief 60-items version of the NEO Personality Inventory-3 (McCrae et al, 2005).…”

Section: Methodsmentioning

confidence: 94%

“…Despite this, FCQ has also received some criticism because it may introduce artificial dependences among the blocks, yielding ipsative or partially ipsative scale scores, and distorting reliability estimates, factorial structure, and criterion-related validity (e.g., Brown and Maydeu-Olivares, 2013). Note that this criticism has largely ignored the fact that ipsativity is a property of the scoring method and the specific task for the respondent, not of the FCQ itself (Morillo et al, 2019). Indeed, Hicks (1970) suggested that three different types of FCQ measures should be distinguished: (1) purely ipsative, (2) quasi-ipsative, and (3) normative.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Controlling for Response Biases in Self-Report Scales: Forced-Choice vs. Psychometric Modeling of Likert Items

et al. 2019

Self Cite

View full text Add to dashboard Cite

One important problem in the measurement of non-cognitive characteristics such as personality traits and attitudes is that it has traditionally been made through Likert scales, which are susceptible to response biases such as social desirability (SDR) and acquiescent (ACQ) responding. Given the variability of these response styles in the population, ignoring their possible effects on the scores may compromise the fairness and the validity of the assessments. Also, response-style-induced errors of measurement can affect the reliability estimates and overestimate convergent validity by correlating higher with other Likert-scale-based measures. Conversely, it can attenuate the predictive power over non-Likert-based indicators, given that the scores contain more errors. This study compares the validity of the Big Five personality scores obtained: (1) ignoring the SDR and ACQ in graded-scale items (GSQ), (2) accounting for SDR and ACQ with a compensatory IRT model, and (3) using forced-choice blocks with a multi-unidimensional pairwise preference model (MUPP) variant for dominance items. The overall results suggest that ignoring SDR and ACQ offered the worst validity evidence, with a higher correlation between personality and SDR scores. The two remaining strategies have their own advantages and disadvantages. The results from the empirical reliability and the convergent validity analysis indicate that when modeling social desirability with graded-scale items, the SDR factor apparently captures part of the variance of the Agreeableness factor. On the other hand, the correlation between the corrected GSQ-based Openness to Experience scores, and the University Access Examination grades was higher than the one with the uncorrected GSQ-based scores, and considerably higher than that using the estimates from the forced-choice data. Conversely, the criterion-related validity of the Forced Choice Questionnaire (FCQ) scores was similar to the results found in meta-analytic studies, correlating higher with Conscientiousness. Nonetheless, the FCQ-scores had considerably lower reliabilities and would demand administering more blocks. Finally, the results are discussed, and some notes are provided for the treatment of SDR and ACQ in future studies.

show abstract

Section: Resultssupporting

confidence: 92%

Section: Methodsmentioning

confidence: 94%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Controlling for Response Biases in Self-Report Scales: Forced-Choice vs. Psychometric Modeling of Likert Items

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is known that forced-choice inventories, especially quasi-ipsative tests, are a preferable option when it comes to control faking or social desirability [ 57 – 61 ]. Furthermore, quasi-ipsative personality inventories have shown a similar or a better predictive validity than personality tests with other formats in the prediction of important criteria in both occupational and educational contexts [ 62 – 64 ]. In the study of CAB, only Cuadrado et al [ 50 ] have analyzed this question using a quasi-ipsative questionnaire for higher education students.…”

Section: Discussionmentioning

confidence: 99%

Individual differences and counterproductive academic behaviors in high school

Cuadrado¹,

Salgado²,

Moscoso³

2020

PLoS ONE

View full text Add to dashboard Cite

Counterproductive academic behaviors (CAB) is a problem that has plagued academic institutions for centuries. However, research has mostly been focused on higher learning institutes in North America. For this reason, literature on CAB must be expanded to other geographical areas and academic levels. The present research analyses the prevalence and correlates of CAB in a sample of Spanish high school students. The results indicate that CAB is a common phenomenon, cheating and low effort behaviors being the most prevalent forms. Correlational analyses revealed that conscientiousness (ρ =-.55, p < .01), emotional stability (ρ = .28, p < .01), and agreeableness (ρ =-.26, p < .05) are predictors of CAB. Multiple regression analyses showed that conscientiousness is the dimension exerting the strongest impact on CAB (β =-.64, p < .01), followed by agreeableness, and emotional stability. These three dimensions accounted for 51% of CAB variance. Last, implications for theory and practice are described.

show abstract

“…Finally, Salgado et al (2019) revisit a current construct in the prediction of professional performance: subjective well-being (SWB). In an extraordinary effort, the authors develop a four-year longitudinal study in which they relate the cognitive and emotional components of SWB with job performance.…”

Section: Assessment Of New Constructs In the Recruitment And Selectiomentioning

confidence: 99%

New Insights on Technology and Assessment: Introduction to JWOP Special Issue

Izquierdo

Aguado

Ponsoda-Gil

2019

Revista De Psicología Del Trabajo Y De Las Organizaciones

View full text Add to dashboard Cite

The Journey from Likert to Forced-Choice Questionnaires: Evidence of the Invariance of Item Parameters

Abstract: Several meta-analyses have shown that the Five-Factor Model of personality predicts a wide range of performance outcomes in the workplace and is a useful framework for organizing most personality measures (see, e.g.

Cited by 31 publications

References 36 publications

Controlling for Response Biases in Self-Report Scales: Forced-Choice vs. Psychometric Modeling of Likert Items

Controlling for Response Biases in Self-Report Scales: Forced-Choice vs. Psychometric Modeling of Likert Items

Individual differences and counterproductive academic behaviors in high school

New Insights on Technology and Assessment: Introduction to JWOP Special Issue

Contact Info

Product

Resources

About