Automating Testing of Visual Observed Concurrency

Dewan, Prasun; Wortas, Andrew; Liu, Zhizhou; George, Samuel D.; Gu, Bowen; Wang, Hao

doi:10.1109/eduhpc54835.2021.00010

Cited by 3 publications

(1 citation statement)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 10 shows the count of tools by different evaluation techniques that involved comparing the results from the AAT with a human and the authors' sentiment of the performance of the AAT. Most are evaluated against the accuracy of the AAT compared to human-provided grades [79,108,166,172,181,185,186,190], with most reporting positive results.…”

Section: Performance Against Human Graders (Rq4 Rq5)mentioning

confidence: 99%

Automated Grading and Feedback Tools for Programming Education: A Systematic Review

Messer,

Brown,

Kölling

et al. 2024

ACM Trans. Comput. Educ.

View full text Add to dashboard Cite

We conducted a systematic literature review on automated grading and feedback tools for programming education. We analysed 121 research papers from 2017 to 2021 inclusive and categorised them based on skills assessed, approach, language paradigm, degree of automation and evaluation techniques. Most papers assess the correctness of assignments in object-oriented languages. Typically, these tools use a dynamic technique, primarily unit testing, to provide grades and feedback to the students or static analysis techniques to compare a submission with a reference solution or with a set of correct student submissions. However, these techniques’ feedback is often limited to whether the unit tests have passed or failed, the expected and actual output, or how they differ from the reference solution. Furthermore, few tools assess the maintainability, readability or documentation of the source code, with most using static analysis techniques, such as code quality metrics, in conjunction with grading correctness. Additionally, we found that most tools offered fully automated assessment to allow for near-instantaneous feedback and multiple resubmissions, which can increase student satisfaction and provide them with more opportunities to succeed. In terms of techniques used to evaluate the tools’ performance, most papers primarily use student surveys or compare the automatic assessment tools to grades or feedback provided by human graders. However, because the evaluation dataset is frequently unavailable, it is more difficult to reproduce results and compare tools to a collection of common assignments.

show abstract