No abstract

Abstract-Achievement badges are a form of gamification that are used in an attempt to increase user engagement and motivation in various systems. A badge is typically a graphical icon that appears as a reward for the user after reaching an achievement but that has no practical value. In this study, we describe and evaluate the use of achievement badges in the TRAKLA2 online learning environment where students solve interactive, automatically assessed exercises in a Data Structures and Algorithms course throughout the semester. We conducted an experiment where the students (N=281) were randomly divided into a treatment and a control group, with and without achievement badges. Students in the treatment group were awarded achievement badges, for example, for solving exercises correctly on the first attempt, doing exercises early, or solving all the exercises in a round with full points. Grading was the same for both groups, i.e. collecting badges did not affect the final grade, even though the exercise points themselves did. Students' activity in TRAKLA2 was logged in order to find out whether the achievement badges had an effect on their behavior. We also collected numerical and open-ended feedback in order to find out students' attitudes towards the badges. Our results show that achievement badges can be used to affect students' behavior. Statistically significant differences were observed in the time used per exercise, number of sessions, total time, and normalized total number of badges. Furthermore, the majority of the students reported being motivated by the badges. Based on our findings, achievement badges seem to be a promising method to motivate students and to encourage desired study practices.

In this study, we examined gamification in relation to achievement goal orientation. Achievement goal orientation is a psychological conceptualization that characterizes students' preferences to different goals, outcomes and rewards. We added achievement badges to an online learning environment used in a Data Structures and Algorithms course (N=278), and examined the responses of students with different achievement goal orientation profiles. Furthermore, we analyzed how students who were most motivated by badges differ from others in terms of achievement goal orientation and behavior.We found no significant differences in the behavior of the different goal orientation groups regarding badges when analyzing the log data from the learning environment. However, their attitudes towards the badges varied. On the other hand, we found that students who reported high motivation towards badges had higher mastery-intrinsic, mastery-extrinsic and performanceapproach orientation, and lower avoidance-orientation than others. All of them were already high-performing before the badges were introduced. However, not all high-performing students were motivated by the badges. We also identified a small avoidanceoriented group who reported very low motivation towards the badges. The results of this paper shed light on the reasons why students respond differently to gamification and on what underlying motivational aspects might contribute to a high or low motivation towards gamification.

In online learning environments where automatic assessment is used, students often resort to harmful study practices such as procrastination and trial-and-error. In this paper, we study two teaching interventions that were designed to address these issues in a university-level computer science course. In the first intervention, we used achievement badges, a form of gamification, to reward students for submitting early, solving exercises with few iterations, and completing the exercises with full points. In the second intervention, we used heatmap visualizations that show a prediction of the student's success if he or she continues to behave in the same way, based on data from earlier courses. The results from a controlled experiment show that the heatmap visualizations had an impact on how early the students submitted their exercises and how many points they earned from them. We also observed that performance approach goal orientation correlated with an interest towards the achievement badges whereas performance avoidance correlated with an interest towards the visualizations. The results indicate that increasing students' awareness of their behavior can have a positive impact on their study practices and results. However, the same type of feedback may not suit all students because of individual differences.

Achievement badges are increasingly used to enhance educational systems and they have been shown to affect student behavior in different ways. However, details on best practices and effective concepts to implement badges from a non-technical point of view are scarce. We implemented badges to our learning management system, used them on a large course and collected feedback from students. Based on our experiences, we present recommendations to other educators that plan on using badges.