Examining Online Course Evaluations and the Quality of Student Feedback

Abstract

The purpose of this article was to provide a comprehensive review of research on the quality of student feedback from post-secondary institutions using online course evaluations versus traditional paper-pencil methods. Nineteen peer-reviewed articles published from 2000 to 2020 were examined for changes to course evaluations following a transition to online collection methods. Three themes emerged from the literature: effects on response rates, presence of non-response bias, and effects on comment quality. Results suggest that using online methods for collecting student feedback tends to decrease response rates somewhat, however, the effect is often temporary. Further, using online methods generated conflicting results on the presence of a non-response bias in open-ended comments with online methods. Many studies demonstrated that online methods increase the word counts in student-provided comments and that the constructive nature of the comments improved as well. The results may inform teaching and policy decisions as more institutions transition to online course evaluation collection methods, particularly given the restrictions imposed by the current COVID-19 crisis. Suggestions for future research include examining the usability of comments as well as trends in student feedback quality following the transition to emergency remote teaching during the global pandemic.

Modeling unit non-response and validity of online teaching evaluation in higher education using generalizability theory approach

“…As such, the majority of the students did not participate in this exercise. This trend of result reflects the outcome of recent studies that have reported unit non-response as a common phenomenon in online surveys ( Marcham et al, 2020 ; Čehovin et al, 2022 ; Falk and Thies, 2022 ; Plante et al, 2022 ). Nevertheless, the recorded high unit non-response rates in this context cannot be entirely attributed to the transition from a paper-and-pencil survey to an online survey.…”

“…This transition has also led to decreased rates of student participation, resulting in high rates of non-response ( Dillman et al, 2002 ). Several studies have shown that unit non-response (i.e., individuals not responding to any item on the survey instrument or complete non-participation) in online evaluation surveys is a key challenge for the introduction of technology in students’ appraisal of teaching and learning ( Guder and Malliaris, 2013 ; Marcham et al, 2020 ; Čehovin et al, 2022 ; Falk and Thies, 2022 ; Plante et al, 2022 ).…”

“…These previous studies have stressed that the sources of variation in students’ appraisal of teaching are attributed to raters (i.e., students), items, occasions, course types, and teacher characteristics unrelated to teaching. Meanwhile, the increasing rate of non-response to online teaching evaluation survey in higher education have been argued to also influence the validity of teaching evaluations ( Adams and Umbach, 2012 ; Guder and Malliaris, 2013 ; Marcham et al, 2020 ; Čehovin et al, 2022 ; Falk and Thies, 2022 ; Plante et al, 2022 ).…”

Features of the organization of distance learning in the process of training specialists in administration and public administration (Polish experience)

“…E-learning Satisfaction Questionnaire, ELSQ (Cofini, et al, 2022) Course evaluation (Kuch & Roberts, 2018) End-Of-Course Evaluations, EOCE (Plante et al, 2022) Students 'Evaluation of Educational Quality, SEEQ (Fatani, 2020) Limitations: monocriteria, qualitative presentation of data, contextuality (program-or course-driven), subjective method of diagnosis.…”