On the Speed Sensitivity Parameter in the Lognormal Model for Response Times and Implications for High-Stakes Measurement Practice

Abstract

In high-stakes testing, often multiple test forms are used and a common time limit is enforced. Test fairness requires that ability estimates must not depend on the administration of a specific test form. Such a requirement may be violated if speededness differs between test forms. The impact of not taking speed sensitivity into account on the comparability of test forms regarding speededness and ability estimation was investigated. The lognormal measurement model for response times by van der Linden was compared with its extension by Klein Entink, van der Linden, and Fox, which includes a speed sensitivity parameter. An empirical data example was used to show that the extended model can fit the data better than the model without speed sensitivity parameters. A simulation was conducted, which showed that test forms with different average speed sensitivity yielded substantial different ability estimates for slow test takers, especially for test takers with high ability. Therefore, the use of the extended lognormal model for response times is recommended for the calibration of item pools in high-stakes testing situations. Limitations to the proposed approach and further research questions are discussed.

Differences in Discrimination with Respect to Latent Trait and Test-Taking Speed Across Items Measuring Sub-Domains of Cognitive Ability

“…When it comes to comparing item difficulty and time consumption requirements, the results are more straightforward—difficult items (scientific reasoning and mathematical reasoning) tend to be the ones that also require more time consumption, reflected in the positive correlation between b and λ parameters (Table 5). This has implications for high-stakes testing, as in the case of identifying gifted students for special programs, because inadequate test administration time can impact students’ response times to difficult items and differentially disadvantage slow versus fast test-takers (Becker, Debeer, Weirich, & Goldhammer, 2021). Our findings suggest that adjusting test administration time to match item difficulty and take into account domains that are more time-intensive can improve fairness with respect to students across different test-taking speeds.…”

A classification model for continuous responses: Identifying risk perception groups on health‐related activities

“…As pointed out by a reviewer, the responses are in the range of 0–100 which makes the log‐normal model a problematic assumption. However, we consider that the application can yet illustrate the use of the model since that bounded data are a common characteristic of psychological and educational data using tests where the number of responses is defined in a time also limited and log‐normal distribution are commonly assumed (see, e.g., Becker et al., 2021).…”

Controlling the Speededness of Assembled Test Forms: A Generalization to the Three‐Parameter Lognormal Response Time Model

“…A straightforward extension of the two‐parameter lognormal response time model, namely the three‐parameter lognormal response time model, already exists (Fox et al., 2007; Klein Entink et al., 2009; Ranger & Ortner, 2012). Indeed, empirical evidence suggests that the assumption of the two‐parameter lognormal model that all items are equally speed sensitive seems to be unrealistic in practice (Becker et al., 2021). Furthermore, Becker et al.…”