Comparing test quality measures for assessing student-written tests

Edwards, Stephen H.; Shams, Zalia

doi:10.1145/2591062.2591164

Cited by 26 publications

(21 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of automated assessment tools [12,5,20,6] and mutation analysis techniques [1,19] have been examined as a way to assess the quality of student written tests. Most of these techniques measure 'code coverage ability' of student-written tests, but recent work has shown that such tools might produce an overestimation of studentwritten test quality [7].…”

Section: Related Workmentioning

confidence: 99%

“…Some have suggested that the quality of a test can be assessed by its ability to detect bugs or faults in a software program rather than by looking at its coverage of execution paths [7]. One technique for measuring the quality of testing is the 'all-pairs' technique proposed by Goldwasser in 2002 [10] and further developed by Edwards et al [9,7].…”

Section: Related Workmentioning

confidence: 99%

“…One technique for measuring the quality of testing is the 'all-pairs' technique proposed by Goldwasser in 2002 [10] and further developed by Edwards et al [9,7]. Using this approach, all student tests are run against all student programs.…”

Section: Related Workmentioning

confidence: 99%

“…As such, software testing has become an integral part of computer science (CS) curricula in recent years [14,4]. Researchers have extensively examined students' test-writing skills [16,19,7] and the primary focus has been on quantifying the quality of students' test codes, measuring their bug-revealing capabilities, and developing techniques to assess student-written software tests. Secure coding and software security testing, on the other hand, are relatively recent areas of exploration within the pedagogical community.…”

Section: Introductionmentioning

confidence: 99%

“…Using common techniques employed by the pedagogical community, such as the 'all-pair method ' [10,9,7], we explored students' ability to write resilient defensive programs, effective attack programs, and the relationship between the two skills. We observed the effectiveness of the security mindset approach on the overall performance of our students.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Can the Security Mindset Make Students Better Testers?

Hooshangi

Weiss

Cappos

2015

Proceedings of the 46th ACM Technical Symposium on Computer Science Education

View full text Add to dashboard Cite

Writing secure code requires a programmer to think both as a defender and an attacker. One can draw a parallel between this model of thinking and techniques used in test-driven development, where students learn by thinking about how to effectively test their code and anticipate possible bugs. In this study, we analyzed the quality of both attack and defense code that students wrote for an assignment given in an introductory security class of 75 (both graduate and senior undergraduate levels) at NYU. We made several observations regarding students' behaviors and the quality of both their defensive and offensive code. We saw that student defensive programs (i.e., assignments) are highly unique and that their attack programs (i.e., test cases) are also relatively unique. In addition, we examined how student behaviors in writing defense programs correlated with their attack program's effectiveness. We found evidence that students who learn to write good defensive programs can write effective attack programs, but the converse is not true. While further exploration of causality is needed, our results indicate that a greater pedagogical emphasis on defensive security may benefit students more than one that emphasizes offense.

show abstract