Gamifying a Software Testing Course with Code Defenders

Fraser, Gordon; Gambi, Alessio; Kreis, Marvin; Rojas, José Miguel

doi:10.1145/3287324.3287471

Cited by 49 publications

(20 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a source of example programs and tests, we used nine Java classes contained in a dataset originating from experiments with Code Defenders [7], an educational testing game (Table I). Code Defenders is a mutation testing game where users compete to write faulty mutant variants of a particular program and unit tests to detect these mutants, and has been shown to support the crowdsourcing of mutation testing by Rojas et al [7], as well as the teaching of software testing techniques by Fraser et al [6]. The subject classes are taken from real open-source Java projects, with the exception of Lift, which was written as a simple introduction to Code Defenders [6].…”

Section: B Subject Programsmentioning

confidence: 99%

“…Code Defenders is a mutation testing game where users compete to write faulty mutant variants of a particular program and unit tests to detect these mutants, and has been shown to support the crowdsourcing of mutation testing by Rojas et al [7], as well as the teaching of software testing techniques by Fraser et al [6]. The subject classes are taken from real open-source Java projects, with the exception of Lift, which was written as a simple introduction to Code Defenders [6]. Importantly for our setting, each of the Java classes comes with a large number of student-written test cases, which we use as a source for grading test suites.…”

Section: B Subject Programsmentioning

confidence: 99%

See 1 more Smart Citation

The Influence of Test Suite Properties on Automated Grading of Programming Exercises

Clegg

McMinn

2020

2020 IEEE 32nd Conference on Software Engineering Education and Training (CSEE&T)

Self Cite

View full text Add to dashboard Cite

Automated grading allows for the scalable assessment of large programming courses, often using test cases to determine the correctness of students' programs. However, test suites can vary in multiple ways, such as quality, size, and coverage. In this paper, we investigate how much test suites with varying properties can impact generated grades, and how these properties cause this impact. We conduct a study on artificial faulty programs that simulate students' programming mistakes and test suites generated from manually written tests. We find that these test suites generate greatly varying grades, with the standard deviation of grades for each fault typically representing ∼84% of the grades not apportioned to the fault. We show that different properties of test suites can influence the grades that they produce, with coverage typically making the greatest effect, and mutation score and the potentially redundant repeated coverage of lines also having a significant impact. We offer suggestions based on our findings to assist tutors with building grading test suites that assess students' code in a fair and consistent manner. These suggestions include ensuring that test suites have 100% coverage, avoiding unnecessarily recovering lines, and checking test suites using real or artificial faults.

show abstract

Section: B Subject Programsmentioning

confidence: 99%

Section: B Subject Programsmentioning

confidence: 99%

The Influence of Test Suite Properties on Automated Grading of Programming Exercises

Clegg

McMinn

2020

2020 IEEE 32nd Conference on Software Engineering Education and Training (CSEE&T)

Self Cite

View full text Add to dashboard Cite

show abstract

“…The results showed that CODE DEFENDERS generated greater coverage and its mutants are more difficult to kill. More recently, a study explored the use of the CODE DEFENDERS in the classroom (Fraser et al, 2019). The results pointed to students more engaged and satisfied with the use of the tool.…”

Section: Related Workmentioning

confidence: 99%

Is It Worth Using Gamification on Software Testing Education? An Extended Experience Report in the Context of Undergraduate Students

Jesus

Ferrari

Paschoal

et al. 2020

JSERD

View full text Add to dashboard Cite

Context: Testing is fundamental in the software development process. Nevertheless, testing education faces the key challenge of ensuring that undergraduate students acquire knowledge and skills they need for their future careers by matching what is taught in the classroom to industry standards. In this context, gamification can be used as an alternative educational approach. It uses game elements in real-world contexts in order to increase people's motivation and engagement in tasks that require external stimuli, especially in educational contexts.Objective: Reporting on results of an experimental study designed to assess the impact of gamification on software testing education, as well as reporting on the experience of building a supporting gamified platform. Method: We performed a systematic literature mapping aiming at characterizing how gamification has been explored in the software testing context. In addition, some of the problems faced by testing education were identified through an ad-hoc search. Then, we developed a gamified approach and a platform that have been used to run five 4-hour functional testing classes with undergraduate students from four Brazilian institutions of higher education.Results: Overall, students who learned with the traditional approach felt more motivated than those who learned with the gamified approach, whereas the performance of both groups were similar. On the other hand, feedback questionnaires indicated that the gamified class was more attractive (in terms of attention) and funny. Moreover, we observed that building a gamified platform is complex and challenging, particularly for the definition of the game mechanics and dynamics.Conclusion: Even though the results in terms of students' performance was neutral, and the motivation of students did not increase due to the introduction of gamification, the experience of having used this alternative approach is considered positive, as it provided a more enjoyable and funny environment, both from the researcher's and students' point of view. Furthermore, with this experience we foresee we can do better in terms of gamified teaching in future work.

show abstract

“…A gamification strategy was proposed by Fraser et al [Fraser et al 2019], to integrate Code Defenders game as a semester-long activity of an undergraduate and graduate course on software testing. Students compete over code under test by either introducing faults ("attacking") or by writing tests ("defending") to reveal these faults.…”

Section: Introductionmentioning

confidence: 99%

“…We used common knowledge (e.g., complementary use of testing techniques) to motivate students about the importance of testing their code. Other courses try to innovate the way testing can be taught [Smith et al 2012, Fraser et al 2019, Paschoal et al 2019. We took a step to innovate by using a simple strategy and work more in the students' perceptions of the need for testing.…”

Section: Introductionmentioning

confidence: 99%

Using an Incremental Testing Strategy to Improve Students’ Perception of Software Quality

Santos

Mori

Souza

2021

Anais Do XXIX Workshop Sobre Educação Em Computação (WEI 2021)

View full text Add to dashboard Cite

Teaching software testing should include a broad view of the main techniques, criteria, and tools. In general, students have few opportunities to test their code-projects suitably during the undergraduate course and, therefore, teaching software testing in practice is crucial to students recognize the advantages and limitations of different testing techniques. This paper reports the experience of teaching software testing in practice, with students applying an incremental testing strategy to validate their software projects. Students selected a software project developed during their undergraduate and an incremental testing strategy, including testing criteria learned in the class. The students should choose the testing techniques, apply them and write a report with the results and perceptions. Through this experience, it was possible to show to the students, in practice, the importance of combining more than one technique during the software testing activity.

show abstract

Gamifying a Software Testing Course with Code Defenders

Cited by 49 publications

References 10 publications

The Influence of Test Suite Properties on Automated Grading of Programming Exercises

The Influence of Test Suite Properties on Automated Grading of Programming Exercises

Is It Worth Using Gamification on Software Testing Education? An Extended Experience Report in the Context of Undergraduate Students

Using an Incremental Testing Strategy to Improve Students’ Perception of Software Quality

Contact Info

Product

Resources

About