The Impact of an Audience Response System on a summative Assessment, a controlled Field Study

Abstract

Background: Audience response systems allow to activate the audience and to receive a direct feedback of participants during lectures. Modern systems do not require any proprietary hardware anymore. Students can directly respond on their smartphone. Several studies reported about a high level of satisfaction of students when audience response systems are used, however their impact on learning success is still unclear.Methods: In order to evaluate the impact of an audience response system on the learning success we implemented the audience response system eduVote into a seminar series and performed a controlled crossover study on its impact on assessments. 154 students in nine groups were taught the same content. In four groups, eduVote was integrated for the first topic while five groups were taught this topic without the audience response systems. For a second topic, the groups were switched: Those groups who were taught before using eduVote were now taught without the audience response system and vice versa. We then analysed the impact of the audience response system on the students’ performance in a summative assessment and specifically focused on questions dealing with the topic, for which the audience response system was used during teaching. We further assessed the students’ perception on the use of eduVote using questionnaires.Results: In our controlled crossover study we could not confirm an impact of the audience response system eduVote on long-term persistence i.e. the students’ performance in the summative assessment. Our evaluation revealed that students assessed the use of eduVote very positively, felt stronger engaged and better motivated to deal with the respective topics and would prefer their integration into additional courses as well. In particular we identified that students who feel uncomfortable with answering questions in front of others profit from the use of an audience response system during teaching.Conclusions: Audience response systems motivate and activate students and increase their engagement during classes. However, their impact on long-term persistence and summative assessments may be limited. Audience response systems, however, specifically allow activating students which cannot be reached by the traditional way of asking questions without such an anonymous tool.

A Sandwich‐model experiment with personal response systems on epigenetics: insights into learning gain, student engagement and satisfaction

“…Interestingly, a study which compared PRS vs. non‐PRS within the peer instruction model reports that clickers may seem at first instance motivating but no significant difference was detected in the conceptual learning gain, and therefore, peer instruction is viewed as an effective but clicker‐independent instructional approach [53]. Furthermore, a recent German study [54] reports positive students’ attitudes towards PRS, higher motivation and confidence as well as a request for PRS integration in more courses. However, their data could not confirm a long‐term positive effect on assessment results implying that PRS effect on actual learning gain could be less profound than its marketing profile and popularity of usage.…”

Use of an audience response system during a continuing education action about oral lesion diagnosis

“…had already shown that participants reported a positive classroom perception and considered Socrative a helpful tool to promote more active participation in classes, compared to other ARS systems. Other studies also found that the use of an ARS tool obtained positive assessments as well as fostered classroom interaction and participation 10,16,25 …”

“…Active learning engages students, may assist in long‐term knowledge retention, and is more effective in the development of thinking skills than traditional lectures 23,24 . The use of learning technologies is an option to foster students’ participation in classes and create a proper environment for active learning 23,25 . In our study, participants considered that technology can improve the learning experience and should be utilized more in classes.…”

“…Yes (%) No (%) use of learning technologies is an option to foster students' participation in classes and create a proper environment for active learning. 23,25 In our study, participants considered that technology can improve the learning experience and should be utilized more in classes. They also believe the use of educational technology improves awareness levels during classes.…”

“…Other studies also found that the use of an ARS tool obtained positive assessments as well as fostered classroom interaction and participation. 10,16,25 OMSpec and DentStu performed better than Hyg on questions about the nature of oral lesions, probably because they had more training on oral lesion diagnostics and/or recent contact with the topic. Curiously, there was no difference between the performance of dentists and dental hygienists on this question.…”