Animated Process-Transparency in Student Evaluation of Teaching: Effects on the Quality and Quantity of Student Feedback

Abstract

Low response rate and quality are common challenges when administrating student evaluations of teaching (SETs). This study aimed to increase SET response rate and quality by investigating a widely recommended strategy to overcome an underlying reason for students' non-responsiveness, namely the lack of transparency about SET processes. In Study 1, we tested first-year students' faith in SET before and after watching an animation that illustrated the SET process. Study 2 consisted of a quasi-experiment in an authentic setting among third-year students in which faith in SET, response rates and response quality were compared between students who were shown the animation versus a control group. Results showed that using an animation to increase transparency of SET processes increased students' faith in SET in both Study (d = 0.60), and Study 2 (d = 0.29). The response rate on the Likert-scale questions in end-of-course surveys increased after the animation (Cramer's V = 0.16), but no significant effects were found on response rates and quality on the open questions in the same survey (all p's > 0.05). Further research and discussion are needed on how, and which, feasible interventions can help to improve students' SET participation, such as offering guidance on how to provide feedback.Higher education institutes worldwide strive to collaborate with students to improve their teaching. To reach this goal, students are asked to assess their education by filling out student evaluations of teaching (SETs). The evaluations are often administered at the end of a course, asking students about teacher effectiveness and the course's strengths and weaknesses. While SETs can provide valuable information for teachers and institutes, the percentage of students who voluntarily fill them in is low (Spooren, Brockx, and Mortelmans 2013; Goos and Salomons 2017), as is the quality of the comments which can be rude or thoughtless (Lakeman et al. 2022a(Lakeman et al. , 2022b(Lakeman et al. , 2022c. A study on the effect of anonymous SET at Australian universities shows 20% of feedback provided through SET to be non-constructive, ranging from allegations to threats (Lakeman et al. 2022a). Such comments can elevate teacher anxiety and stress as well as perceived pressure to compromise the quality of teaching and assessment to increase evaluation results (Lakeman et al. 2022b(Lakeman et al. , 2022c. Mid to long-term effects extend beyond the classroom and may even affect retention and recruitment of teachers (Lakeman et al. 2022c).