Participation and performance on paper- and computer-based low-stakes assessments

Abstract

Background: High-stakes assessments, such the Graduate Records Examination, have transitioned from paper to computer administration. Low-stakes research-based assessments (RBAs), such as the Force Concept Inventory, have only recently begun this transition to computer administration with online services. These online services can simplify administering, scoring, and interpreting assessments, thereby reducing barriers to instructors' use of RBAs. By supporting instructors' objective assessment of the efficacy of their courses, these services can stimulate instructors to transform their courses to improve student outcomes. We investigate the extent to which RBAs administered outside of class with the online Learning About STEM Student Outcomes (LASSO) platform provide equivalent data to tests administered on paper in class, in terms of both student participation and performance. We use an experimental design to investigate the differences between these two assessment conditions with 1310 students in 25 sections of 3 college physics courses spanning 2 semesters. Results: Analysis conducted using hierarchical linear models indicates that student performance on low-stakes RBAs is equivalent for online (out-of-class) and paper-and-pencil (in-class) administrations. The models also show differences in participation rates across assessment conditions and student grades, but that instructors can achieve participation rates with online assessments equivalent to paper assessments by offering students credit for participating and by providing multiple reminders to complete the assessment. Conclusions: We conclude that online out-of-class administration of RBAs can save class and instructor time while providing participation rates and performance results equivalent to in-class paper-and-pencil tests.

Developing a conceptual assessment for a modular curriculum

“…In fall 2017, we adopted an online administration procedure that incorporates best practices for online administration based on the research literature [13]. Each term instructors complete a Qualtrics survey, which provides the start and end dates of the term and the units to be covered.…”

Leveraging understanding of energy from physics to overcome unproductive intuitions in chemistry

“…The surveys were made available for one week, and a small amount of homework credit was awarded for survey completion. Providing a small amount of credit has been shown to help ensure that the performance of the subset of students completing an online assessment is similar to what would be obtained if the assessment were administered on paper in class [49]. All research activities for this study were deemed "exempt" by the Penn State Institutional Review Board, and those students who did not consent to have their data included in the study have been excluded from the analysis.…”

Efficacy and perceptions of assessment digitization within a large‐enrollment mechanical and aerospace engineering course

“…Controlled use of CBA has shown to correlate positively with overall course grades in a 221‐student controlled study in General Chemistry courses at Iowa State University . On the other hand, skill inventories in a study spanning 25 sections of college physics course showed that participation in asynchronous online CBA varied significantly based on the students’ ranking within the course . Thus to increase engagement and rapid remediation for all students uniformly, the testing assessment center‐based approach developed herein promulgates an integrated testing and tutoring methodology .…”