No abstract

In low-stakes testing contexts, there are minimal personal consequences associated with examinee performance. Examples include assessments administered for research, program evaluation, test development, and international comparisons (e.g., Programme for International Student Assessment [PISA]). Because test-taking motivation can suffer in low-stakes conditions, the Student Opinion Scale (SOS) is commonly administered to measure test-taking effort and how personally important the examinee feels it is to do well on the test. Although popular, studies of the scale’s internal validity yield conflicting findings. The present study critically evaluates the creation of the SOS and considers its factor structure across six samples of college students differing in their college experience level and version of the SOS administered. Because findings only support the internal validity of the effort subscale, further study and development of the importance subscale is recommended.

The anticipation, inclusion, responsiveness, and reflexivity (AIRR) framework (Stilgoe et al., 2013) is a novel framework that has helped those in science and technology fields shift their focus from products to the processes used to create those products. However, the framework has not been known to be applied to the development and implementation of data analytics in higher education. In a case study of creating an early-alert retention system at James Madison University, a working group of ~20 faculty, staff, and students creatively utilized the AIRR framework. The present study discusses how the AIRR framework was utilized to observe and enhance group processes, and the outcomes of those enhanced processes.