2018
DOI: 10.31234/osf.io/dbwn8
|View full text |Cite
Preprint
|
Sign up to set email alerts
|

On the Statistical and Practical Limitations of Thurstonian IRT Models

Abstract: Forced-choice questionnaires have been proposed to avoid common response biases typically associated with rating scale questionnaires. To overcome ipsativity issues of trait scores obtained from classical scoring approaches of forced-choice items, advanced methods from item response theory (IRT) such as the Thurstonian IRT model have been proposed. For convenient model specification, we introduce the thurstonianIRT R package, which uses Mplus, lavaan, and Stan for model estimation. Based on practical considera… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1
1

Citation Types

3
46
0

Year Published

2019
2019
2022
2022

Publication Types

Select...
6

Relationship

1
5

Authors

Journals

citations
Cited by 11 publications
(49 citation statements)
references
References 44 publications
3
46
0
Order By: Relevance
“…Nevertheless, the findings these authors present continue to cast doubt on the usefulness of TIRT scoring in applied, high-stakes testing, as the average criterion-related validity presented for even nonwork-related criteria tended to be smaller for TIRT, ultimately in favor of CTT scoring. In fact, recent theoretical and simulation work being conducted by Bürkner, Schulte, and Holling (2018) bring the authors to a very similar conclusion. Taken together with the current study, this highlights the uncertainty in understanding exactly what variance is being captured by TIRT that is unique from CTT.…”
Section: Reconciling Results: Trait Retrievalsupporting
confidence: 53%
“…Nevertheless, the findings these authors present continue to cast doubt on the usefulness of TIRT scoring in applied, high-stakes testing, as the average criterion-related validity presented for even nonwork-related criteria tended to be smaller for TIRT, ultimately in favor of CTT scoring. In fact, recent theoretical and simulation work being conducted by Bürkner, Schulte, and Holling (2018) bring the authors to a very similar conclusion. Taken together with the current study, this highlights the uncertainty in understanding exactly what variance is being captured by TIRT that is unique from CTT.…”
Section: Reconciling Results: Trait Retrievalsupporting
confidence: 53%
“…Among these studies, most have examined the equivalence of dominance-model-based SS and FC formats and found mixed evidence. For example, statement factor loadings were found to differ substantially between FC and SS formats (Ackerman, Donnellan, Roberts, & Fraley, 2016; Dueber, Love, Toland, & Turner, 2019; Guenole et al, 2018; Wetzel, Roberts, Fraley, & Brown, 2016), suggesting that respondents may interpret them differently (Bürkner, Schulte, & Holling, in press). While some have reported moderate to high convergent validity (Brown & Maydeu-Olivares, 2011, 2013; Lee, Joo, & Lee, 2019; Lee, Lee, & Stark, 2018; Usami et al, 2016), others have observed very low convergent validity (Anguiano-Carrasco, MacCann, Geiger, Seybert, & Roberts, 2015; Dueber et al, 2019; Seybert, 2013).…”
Section: Psychometric Equivalence Between Fc and Ssmentioning
confidence: 99%
“…Yet a recent simulation suggests that simply changing the parameter estimation method does not lead to sufficiently reliable estimates for many FC questionnaires (Bürkner et al, 2019). When the simulated questionnaires measured five or fewer traits, the model failed to reach a satisfactory level of measurement precision in all practically relevant conditions.…”
mentioning
confidence: 99%
“…In contrast to T-IRT models that measure few traits, T-IRT models have performed well when measuring 30 traits even with only equally keyed items (Bürkner et al, 2019). Even though this trait number represents the most prominent FC personnel selection test, namely the Occupational Personality Questionnaire (OPQ; Brown & Bartram, 2013), most test constructors seek to develop tests with fewer traits.…”
mentioning
confidence: 99%
See 1 more Smart Citation