Bayesian Psychometric Scaling

Abstract

In educational and psychological studies, psychometric methods are involved in the measurement of constructs, and in constructing and validating measurement instruments. Assessment results are typically used to measure student proficiency levels and test characteristics. Recently, Bayesian item response models received considerable attention to analyze test data and to measure latent variables. Bayesian psychometric modeling allows to include prior information about the assessment in addition to information available in the observed response data. An introduction is given to Bayesian psychometric modeling, and it is shown that this approach is very flexible, provides direct estimates of student proficiencies, and depends less on asymptotic results. Various Bayesian item response models are discussed to provide insight in Bayesian psychometric scaling and the Bayesian way of making psychometric inferences. This is done according to a general multilevel modeling approach, where observations are nested in students and items, and students are nested in schools. Different examples are given to illustrate the influence of prior information, the effects of clustered response data following a PISA study, and Bayesian methods for scale construction.

Bayesian data analysis for newcomers

“…Panel B indicates this property with an arrow from the posterior distribution pointing to a "mind" that produced the data. A few recent examples of Bayesian psychometric models include those reported by Fox, van den Berg, and Veldkamp (2015), J. Corter (2011), Oravecz, Vandekerckhove, andBatchelder (2014), Vandekerckhove 7 The full URL is http:// doingbayesiandataanalysis.blogspot .com/2011/10/bayesian-models-of-mind -psychometric.html and a PDF version is available at https://osf.io/mxyck/.…”

Expert knowledge elicitation using item response theory

“…Note that this process can be very expensive and time demanding, also it carries a lot of uncertainty itself, due to the natural variance among students. Alternatively, as discussed by Fox et al (2015), in a Bayesian framework, instead of treating the parameters as fixed quantities, we assign prior distributions to the item parameters. This eliminates the expensive stage of pre-testing the items and the prior distributions will express the item parameters (discrimination or difficulty) and the uncertainty about them.…”