A semi‐parametric within‐subject mixture approach to the analyses of responses and response times

Molenaar, Dylan; Bolsinova, Maria; Vermunt, Jeroen K.

doi:10.1111/bmsp.12117

Cited by 31 publications

(58 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the simulation study, we compared the performance of the unweighted SW test, the unweighted KS test, and the weighted KS test. Data are simulated according to nine different scenarios, mostly based on Molenaar et al (2018) who found biased modeling results of the hierarchical mixture model in the case of nonnormality. The first three scenarios concern Markov mixture models that include Markov-dependent item states, the next three scenarios concern mixture models with independent item states, and the final three scenarios are generated according to a baseline model that does not include item states (i.e., a static model without mixtures).…”

Section: Methodsmentioning

confidence: 99%

Testing the Within-State Distribution in Mixture Models for Responses and Response Times

Kuijpers¹,

Visser²,

Molenaar

2020

Journal of Educational and Behavioral Statistics

Self Cite

View full text Add to dashboard Cite

Mixture models have been developed to enable detection of within-subject differences in responses and response times to psychometric test items. To enable mixture modeling of both responses and response times, a distributional assumption is needed for the within-state response time distribution. Since violations of the assumed response time distribution may bias the modeling results, choosing an appropriate within-state distribution is important. However, testing this distributional assumption is challenging as the latent within-state response time distribution is by definition different from the observed distribution. Therefore, existing tests on the observed distribution cannot be used. In this article, we propose statistical tests on the within-state response time distribution in a mixture modeling framework for responses and response times. We investigate the viability of the newly proposed tests in a simulation study, and we apply the test to a real data set.

show abstract

Section: Methodsmentioning

confidence: 99%

Testing the Within-State Distribution in Mixture Models for Responses and Response Times

Kuijpers¹,

Visser²,

Molenaar

2020

Journal of Educational and Behavioral Statistics

Self Cite

View full text Add to dashboard Cite

show abstract

“…A related yet conceptually distinct strand of work [12,13,14,15] considers responses as coming from a mixture of response processes. In particular, responses are generated by either "fast" or "slow" processes and the relevant item characteristics are allowed to vary as a function of the process.…”

Section: Psychological Approachesmentioning

confidence: 99%

Variation in Respondent Speed and its Implications: Evidence from an Adaptive Testing Scenario

Domingue¹,

Kanopka²,

Stenhaug³

et al. 2020

Preprint

View full text Add to dashboard Cite

As our ability to collect data about respondents increases, approaches for incorporating ancillary data features such as response time are of heightened interest. Models for response time have been advanced, but relatively limited large-scale empirical investigations have been conducted. We take advantage of a unique and massive dataset—data from computer adaptive administrations of the NWEA MAP Growth assessment in two states consisting of roughly 1/4 billion item responses—containing both item responses plus response times to shed light on emergent features of response time behavior. We focus on two behaviors in particular. The first, response acceleration, is a reduction in response time for responses that occur relatively late on the assessment. We further note that such reductions are heterogeneous as a function of estimated ability (lower ability estimates are associated with larger increases in acceleration) and that reductions in response time on later items lead to reductions in accuracy relative to expectation. We also document variation in interplay between speed and accuracy. In some cases, additional time spent on an item is associated with an increase in accuracy; in other cases, the opposite is true. This finding has potential connections to the nascent literature on different within-person response processes. We argue that our approach may be useful in other settings and that the behaviors observed here should be of interest in other data.

show abstract

“…Unfortunately, approaches that are capable of handling one form of contamination do not work well for the other form of contamination (Alqallaf et al ., ). This is apparent for the mixture models that require strong assumptions about the form of contamination and fail if these assumptions are not met (Molenaar et al ., ; Ranger & Kuhn, ). Mixture models have more limitations.…”

Section: Introductionmentioning

confidence: 99%

“…The responses and response times in the regular response mode are usually modelled with the hierarchical model of van der Linden (). The responses and response times in the irregular response mode are assumed to be faster and are modelled differently; for specific variants of such mixture models see Meyer (), Molenaar, Bolsinova, and Vermunt (), Wang and Xu () and Wang et al . ().…”

Section: Introductionmentioning

confidence: 99%

“…The responses and response times in the regular response mode are usually modelled with the hierarchical model of van der Linden (2007). The responses and response times in the irregular response mode are assumed to be faster and are modelled differently; for specific variants of such mixture models see Meyer (2010), Molenaar, Bolsinova, and Vermunt (2018), Wang and Xu (2015) and Wang et al (2018). One could also use the general mixture models of Molenaar and de Boeck (2018) or Molenaar, Oberski, Vermunt, and De Boeck (2016) for this purpose, although they were proposed in a different context.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Robust estimation of the hierarchical model for responses and response times

Ranger

Wolgast

Kuhn

2018

Brit J Math & Statis

View full text Add to dashboard Cite

Van der Linden's (2007, Psychometrika, 72, 287) hierarchical model for responses and response times in tests has numerous applications in psychological assessment. The success of these applications requires the parameters of the model to have been estimated without bias. The data used for model fitting, however, are often contaminated, for example, by rapid guesses or lapses of attention. This distorts the parameter estimates. In the present paper, a novel estimation approach is proposed that is robust against contamination. The approach consists of two steps. In the first step, the response time model is fitted on the basis of a robust estimate of the covariance matrix. In the second step, the item response model is extended to a mixture model, which allows for a proportion of irregular responses in the data. The parameters of the mixture model are then estimated with a modified marginal maximum likelihood estimator. The modified marginal maximum likelihood estimator downweights responses of test-takers with unusual response time patterns. As a result, the estimator is resistant to several forms of data contamination. The robustness of the approach is investigated in a simulation study. An application of the estimator is demonstrated with real data.

show abstract

A semi‐parametric within‐subject mixture approach to the analyses of responses and response times

Cited by 31 publications

References 53 publications

Testing the Within-State Distribution in Mixture Models for Responses and Response Times

Testing the Within-State Distribution in Mixture Models for Responses and Response Times

Variation in Respondent Speed and its Implications: Evidence from an Adaptive Testing Scenario

Robust estimation of the hierarchical model for responses and response times

Contact Info

Product

Resources

About