Background
With digital technologies, competence assessments can provide process data, such as mouse clicks with corresponding timestamps, as additional information about the skills and strategies of test takers. However, in order to use variables generated from process data sensibly for educational purposes, their interpretation needs to be validated with regard to their intended meaning.
Aims
This study seeks to demonstrate how process data from an assessment of multiple document comprehension can be used to represent sourcing, which summarizes activities for the consideration of the origin and intention of documents. The investigated process variables were created according to theoretical assumptions about sourcing, and systematically tested for differences between persons, units (i.e., documents and items), and properties of the test administration.
Sample
The sample included 310 German university students (79.4% female), enrolled in several bachelor's or master's programmes of the social sciences and humanities.
Methods
Regarding the hierarchical data structure, the hypotheses were analysed with generalized linear mixed models (GLMM).
Results
The results mostly revealed expected differences between individuals and units. However, unexpected effects of the administered order of units and documents were detected.
Conclusions
The study demonstrates the theory‐informed construction of process variables from log‐files and an approach for empirical validation of their interpretation. The results suggest that students apply sourcing for different reasons, but also stress the need of further validation studies and refinements in the operationalization of the indicators investigated.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.