Multiple‐Group Joint Modeling of Item Responses, Response Times, and Action Counts with the Conway‐Maxwell‐Poisson Distribution

Abstract

Multiple group modeling is one of the methods to address the measurement noninvariance issue. Traditional studies on multiple group modeling have mainly focused on item responses. In computer-based assessments, joint modeling of response times and action counts with item responses helps estimate the latent speed and action levels in addition to latent ability. These two new data sources can also be used to further address the measurement noninvariance issue. One challenge, however, is to correctly model action counts which can be underdispersed, overdispersed, or equidispersed in real data sets. To address this, we adopted the Conway-Maxwell-Poisson distribution that accounts for different types of dispersion in action counts and incorporated it in the multiple group joint modeling of item responses, response times, and action counts. Bayesian Markov Chain Monte Carlo method was used for model parameter estimation. To illustrate an application of the proposed model, an empirical data analysis was conducted using the Programme for International Student Assessment (PISA) 2015 collaborative problem-solving items where potential measurement noninvariance issue existed between gender groups. Results indicated that Conway-Maxwell-Poisson model yielded better model fit than alternative count data models such as negative binomial and Poisson models. In addition, response times and action counts provided further information on performance differences between groups.

Every Trait Counts: Marginal Maximum Likelihood Estimation for Novel Multidimensional Count Data Item Response Models with Rotation or l1-Regularization for Simple Structure

“…The Conway Maxwell Poisson Model (CMPCM;Forthmann et al, 2020) has no discrimination parameters like a Rasch model, while the Two Parameter Conway Maxwell Poisson Model (2PCMPCM; Beisemann, 2022) includes discrimination parameters. Qiao, Jiao, and He (2023) propose a CMP-based joint modeling approach. Tutz (2022) provides an alternative approach all together for dispersion handling.…”

Exploring the relationship between process data and contextual variables among Scandinavian students on PISA 2012 mathematics tasks

“…Other studies have addressed joint modelling by including not only the number of actions but also even more complicated features, such as action sequences, time, response, and background variables (Han et al, 2019;Tang et al, 2020;Ulitzsch et al, 2021). Qiao et al, (2022) conducted a multigroup joint model with the response, response time, and action sequence, which focused on the group invariance of gender. However, rare studies yet showed the framework of cross-country comparison of latent variables measured by the processing indicators together with the measurement invariance examination for processing data in an international large-scale survey.…”

External Correlates of Adult Digital Problem-Solving Process

“…Studies have demonstrated the utility of process data for a multitude of practical tasks: To start, process data can provide additional information on the measured proficiency or skills, allowing better measurement via process-incorporated scoring rules (Zhang et al, 2023) and process-based measurement models, which typically associate continuous latent proficiency (Chen, 2020; Han et al, 2022; LaMar, 2018; Liu et al, 2018; Xiao & Liu, 2024) or discrete latent skill mastery (Zhan & Qiao, 2022; Liang et al, 2022) with examinees’ choices of correct/incorrect subsequent actions, observed action subsequences, or sequence length. Furthermore, analyses of behavioral characteristics associated with successful/unsuccessful final performance (e.g., Gao, Cui, et al, 2022; Gao, Zhai, et al, 2022; Greiff et al, 2015; He & von Davier, 2016; Qiao & Jiao, 2018; Qiao et al, 2023; Ulitzsch et al, 2021, 2023) can inform test validation and automated scoring. Exploratory analyses of action sequences or sequence-derived patterns, often with cluster analysis (Eichmann et al, 2020; He et al, 2019; Gao, Cui, et al, 2022; Gao, Zhai, et al, 2022; He, Borgonovi, & Suárez-Álvarez, 2023; Hao & Mislevy, 2019; Ulitzsch et al, 2022) or with topic modeling of actions or subsequences (Fang & Ying, 2020; Xu et al, 2018), have revealed different behavioral prototypes among the examinees as they face the same task, providing insights on how individuals navigate and approach computerized tests, digital platforms encountered in daily life, collaborative problems, etc.…”