A Bayesian Item Response Model for Examining Item Position Effects in Complex Survey Data

Trendtel, Matthias; Robitzsch, Alexander

doi:10.3102/1076998620931016

Cited by 6 publications

(5 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is of particular interest whether the Mislevy-Wu model (MM1 and MM2) outperforms other treatments of missing item responses such as the scoring as wrong (model MW) and latent ignorable (models MO1 and MO2). The Bayesian information criterion (BIC) is used for conducting model comparisons ([ 33 ]; see also [ 16 , 120 , 121 , 139 ] for similar model comparisons in PISA, but [ 140 , 141 , 142 ] for improved information criteria in complex surveys). Moreover, the Gilula–Haberman penalty (GHP; [ 143 , 144 , 145 ]) is used as an effect size that is relatively independent of the sample size and the number of items.…”

Section: Pisa 2018 Mathematics Case Study: Methodsmentioning

confidence: 99%

On the Treatment of Missing Item Responses in Educational Large-Scale Assessment Data: An Illustrative Simulation Study and a Case Study Using PISA 2018 Mathematics Data

Robitzsch

2021

EJIHPE

Self Cite

View full text Add to dashboard Cite

Missing item responses are prevalent in educational large-scale assessment studies such as the programme for international student assessment (PISA). The current operational practice scores missing item responses as wrong, but several psychometricians have advocated for a model-based treatment based on latent ignorability assumption. In this approach, item responses and response indicators are jointly modeled conditional on a latent ability and a latent response propensity variable. Alternatively, imputation-based approaches can be used. The latent ignorability assumption is weakened in the Mislevy-Wu model that characterizes a nonignorable missingness mechanism and allows the missingness of an item to depend on the item itself. The scoring of missing item responses as wrong and the latent ignorable model are submodels of the Mislevy-Wu model. In an illustrative simulation study, it is shown that the Mislevy-Wu model provides unbiased model parameters. Moreover, the simulation replicates the finding from various simulation studies from the literature that scoring missing item responses as wrong provides biased estimates if the latent ignorability assumption holds in the data-generating model. However, if missing item responses are generated such that they can only be generated from incorrect item responses, applying an item response model that relies on latent ignorability results in biased estimates. The Mislevy-Wu model guarantees unbiased parameter estimates if the more general Mislevy-Wu model holds in the data-generating model. In addition, this article uses the PISA 2018 mathematics dataset as a case study to investigate the consequences of different missing data treatments on country means and country standard deviations. Obtained country means and country standard deviations can substantially differ for the different scaling models. In contrast to previous statements in the literature, the scoring of missing item responses as incorrect provided a better model fit than a latent ignorable model for most countries. Furthermore, the dependence of the missingness of an item from the item itself after conditioning on the latent response propensity was much more pronounced for constructed-response items than for multiple-choice items. As a consequence, scaling models that presuppose latent ignorability should be refused from two perspectives. First, the Mislevy-Wu model is preferred over the latent ignorable model for reasons of model fit. Second, in the discussion section, we argue that model fit should only play a minor role in choosing psychometric models in large-scale assessment studies because validity aspects are most relevant. Missing data treatments that countries can simply manipulate (and, hence, their students) result in unfair country comparisons.

show abstract

Section: Pisa 2018 Mathematics Case Study: Methodsmentioning

confidence: 99%

On the Treatment of Missing Item Responses in Educational Large-Scale Assessment Data: An Illustrative Simulation Study and a Case Study Using PISA 2018 Mathematics Data

Robitzsch

2021

EJIHPE

Self Cite

View full text Add to dashboard Cite

show abstract

“…The average number of students per item varied across countries between 1337.7 and 2261.3 (M = 1628.0, SD = 273.4). Sampling weights were not taken into account in the analysis because the two-stage stratified clustered sampling design would require a modified computation of the Akaike information criterion (AIC; [45,46]).…”

Section: Resultsmentioning

confidence: 99%

Regularized Generalized Logistic Item Response Model

Robitzsch

2023

Information

Self Cite

View full text Add to dashboard Cite

Item response theory (IRT) models are factor models for dichotomous or polytomous variables (i.e., item responses). The symmetric logistic or probit link functions are most frequently utilized for modeling dichotomous or polytomous items. In this article, we propose an IRT model for dichotomous and polytomous items using the asymmetric generalistic logistic link function that covers a lot of symmetric and asymmetric link functions. Compared to IRT modeling based on the logistic or probit link function, the generalized logistic link function additionally estimates two parameters related to the asymmetry of the link function. To stabilize the estimation of item-specific asymmetry parameters, regularized estimation is employed. The usefulness of the proposed model is illustrated through simulations and empirical examples for dichotomous and polytomous item responses.

show abstract

“…Applications and extensions of WP methods, summarized in Table 2 span multiple substantive domains, including sexual health (Bastos et al, 2018; Kunihama et al, 2016, 2019), environmental health (Vedensky et al, 2022), physical and mental health (Aliverti & Medicine, 2022; Parker et al, 2022; Parker & Holan, 2022; Trendtel & Robitzsch, 2021; Vedensky et al, 2022; Williams & Savitsky, 2021), poverty and housing (Fourrier‐Nicolai & Lubrano, 2020; Gunawan et al, 2020), and employment (Savitsky et al, 2022; Savitsky & Srivastava, 2018; Savitsky & Toth, 2016; Sun et al, 2022). Methodologically, applications and extensions of the method can be broadly classified into regression models, hierarchical models, mixture models, and computational and privacy guards.…”

Section: Weighted Pseudo‐likelihood For Complex Parametric Modelsmentioning

confidence: 99%

Bayesian estimation methods for survey data with potential applications to health disparities research

Wu,

Stephenson

2023

WIREs Computational Stats

View full text Add to dashboard Cite

Understanding how and why certain communities bear a disproportionate burden of disease is challenging due to the scarcity of data on these communities. Surveys provide a useful avenue for accessing hard‐to‐reach populations, as many surveys specifically oversample understudied and vulnerable populations. When survey data is used for analysis, it is important to account for the complex survey design that gave rise to the data, to avoid biased conclusions. The field of Bayesian survey statistics aims to account for such survey design while leveraging the advantages of Bayesian models, which can flexibly handle sparsity through borrowing of information and provide a coherent inferential framework to easily obtain variances for complex models and data types. For these reasons, Bayesian survey methods seem uniquely well‐poised for health disparities research, where heterogeneity and sparsity are frequent considerations. This review discusses three main approaches found in the Bayesian survey methodology literature: (1) multilevel regression and post‐stratification, (2) weighted pseudolikelihood‐based methods, and (3) synthetic population generation. We discuss the advantages and disadvantages of each approach, examine recent applications and extensions, and consider how these approaches may be leveraged to improve research in population health equity.This article is categorized under: Applications of Computational Statistics > Clinical Trials Statistical and Graphical Methods of Data Analysis > Bayesian Methods and Theory Applications of Computational Statistics > Health and Medical Data/Informatics Statistical and Graphical Methods of Data Analysis > Sampling

show abstract

A Bayesian Item Response Model for Examining Item Position Effects in Complex Survey Data

Cited by 6 publications

References 32 publications

On the Treatment of Missing Item Responses in Educational Large-Scale Assessment Data: An Illustrative Simulation Study and a Case Study Using PISA 2018 Mathematics Data

On the Treatment of Missing Item Responses in Educational Large-Scale Assessment Data: An Illustrative Simulation Study and a Case Study Using PISA 2018 Mathematics Data

Regularized Generalized Logistic Item Response Model

Bayesian estimation methods for survey data with potential applications to health disparities research

Contact Info

Product

Resources

About