An Automated System for Interactively Learning Software Testing

Abstract

Testing is an important, time-consuming, and often difficult part of the software development process. It is therefore critical to introduce testing early in the computer science curriculum, and to provide students with frequent opportunities for practice and feedback. This paper presents an automated system to help introductory students learn how to test software. Students submit test cases to the system, which uses a large corpus of buggy programs to evaluate these test cases. In addition to gauging the quality of the test cases, the system immediately presents students with feedback in the form of buggy programs that nonetheless pass their tests. This enables students to understand why their test cases are deficient and gives them a starting point for improvement. The system has proven effective in an introductory class: students that trained using the system were later able to write better test cases-even without any feedback-than those who were not. Further, students reported additional benefits such as improved ability to read code written by others and to understand multiple approaches to the same problem.

Django Web Framework Software Metrics Measurement Using Radon and Pylint

“…McCabe's metric provides the upper bound on module complexity. The graph and formula below define Cyclomatic Complexity [18], [19]. The Abstract Syntax Tree (AST) tree of a Python program to compute Cyclomatic Complexity is analyzed by Radon.…”

Mutation Testing and Self/Peer Assessment: Analyzing their Effect on Students in a Software Testing Course

“…Both mutation testing and peer testing are two of the strategies highlighted in the software testing teaching literature to enhance students' motivation [4], [6]. Mutation testing poses more demanding testing criteria than traditional coverage-based metrics [7], [8] -which usually overestimate the apparent test quality-, and allow students to directly observe examples of bugs that their tests are not able to detect [9]. As for peer testing, it exposes students to different solutions and offers the opportunity to share their knowledge with their classmates, engaging them in the learning process and increasing the quality of their developments [10], [11], [12].…”

The Significance of Positive Verification in Unit Test Assessment

“…Accordingly, educators have been exploring how to both effectively teach and evaluate testing in computer science classes. Pedagogical tools [6,13,19] and approaches [7,12] have been developed to help introduce students to testing throughout the curricula.…”

“…Automated assessment of students' tests (autograders) typically use coverage, a measurement of how much code has been executed by their tests. Otherwise, some tools and studies evaluate students' tests with a concentration on their ability to identify bugs, as accomplished by failing known faulty implementations [11,13,19]. However, there has neither been thorough discussion nor evaluation of how well students' tests accurately confirm acceptable solutions.…”

“…There are also eLearning tools specifically for fostering testing skills. Smith, et al created an automated tool that evaluates students' tests against a large dataset of known buggy solutions and provides feedback based on how well a test fails those bugs [19]. However, like both all-pairs analysis and mutation testing, their tool concentrates on the ability to fail bugs and does not measure rates of positive verification.…”