Editorial: Process Data in Educational and Psychological Measurement

Abstract

The increasing use of computer-based testing and learning environments is leading to a significant reform on the traditional form of measurement, with tremendous extra available data collected during the process of learning and assessment (Bennett et al., 2007(Bennett et al., , 2010. It means that we can learn and describe the respondents' performances not only by their responses, but also their responding processes, in addition to the response accuracy in the traditional tests (Ercikan and Pellegrino, 2017).The recent advances in computer technology enhance the convenient collection of process data in computer-based assessment. One such example is time-stamped action data in an innovative item which allow for the interaction between a respondent and the item. When a respondent attempts an interactive item, his/her actions are recorded, in the form of an ordered sequence of multi-type, time-stamped events. These sorts of data stored in log files, referred to as process data in this book, provide information beyond response data that typically show response accuracy only. This additional information holds promise to help us understand the strategies that underlie test performance and identify key actions that lead to success or failure of answering an item (e.g.,

Personalizing Large‐Scale Assessment in Practice

“…Existing and developing technology can be used to measure and store data on contextual factors that can be incorporated into measurement models. There is a growing body of literature that includes conceptualizing the incorporation of contextual factors into measurement models (Bergner & von Davier, 2019;Mislevy, 2018;Provasnik, 2021), developing empirical methods for making use of personalized data in largescale assessment (e.g., automatic item generation, Gierl & Haladyna, 2012; process data capture and analysis, Jiao et al, 2021;and computational psychometrics, von Davier et al, 2021), and taking an argument-based approach to fairness when artificial intelligence systems are used in assessment (Huggins-Manley et al, 2022).…”

Process and Product in Computer-Based Assessments

“…Newton asked if we were on the "brink of a revolution" involving the use of process data in validation research and to what extent it might become part of the routine practice of validation research (2019, p. 246). The subsequent rapid growth of literature on the uses of process data in assessment demonstrates that this indeed was the case (Goldhammer et al, 2020;Jiao et al, 2021). Clearly, data on assessment response processes from digital probes and sensors (including log files) are now central to digital assessment design, even if the field is not yet entirely sure about the kinds of measurement opportunities they involve.…”

“…That is not to say that sources of process data should not be used for assessment purposes. On the contrary, their potential to inform and improve digital assessments is clear (Goldhammer et al, 2020; Jiao et al, 2021). However, in the absence of systematic and routinized validation, there is a requirement for more explicit arguments and evidence than we might consider necessary for product-based scores (Goldhammer et al, 2021).…”

Understanding the log file data from educational and psychological computer-based testing: A scoping review protocol

“…The application of log file data offers promising advancements in the field of psychological and educational measurement. Specifically, log file data records the response processes of test takers, offering insights into their test performance, behavior patterns, and cognitive strategies [ 4 ]. This rich information goes beyond simple score outcomes, enabling a more detailed interpretation of test-taker proficiency.…”