Automatic Grading and Feedback using Program Repair for Introductory Programming Courses

Parihar, S. J.; Dadachanji, Ziyaan; Singh, Prince Kumar; Das, Rajdeep; Karkare, Amey; Bhattacharya, Arnab

doi:10.1145/3059009.3059026

Cited by 72 publications

(41 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Synthesizing the literature on feedback, we classified common feedback into three types in the context of computing education: [22], [25] All types of feedback were implemented per test case. To effectively implement Hint, we adopted the approach used by Parihar et al [20]. We collected student submissions of the same programming assignments over the last three quarters, summarized the common mistakes and problems, and designed adaptive hints for the top five common mistakes per test case.…”

Section: B System Implementation and Feedback Designmentioning

confidence: 99%

Investigating the Essential of Meaningful Automated Formative Feedback for Programming Assignments

Hao

Wilson

Ottaway

et al. 2019

2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)

View full text Add to dashboard Cite

This study investigated the essential of meaningful automated feedback for programming assignments. Three different types of feedback were tested, including (a) What's wrong -what test cases were testing and which failed, (b) Gapcomparisons between expected and actual outputs, and (c) Hinthints on how to fix problems if test cases failed. 46 students taking a CS2 participated in this study. They were divided into three groups, and the feedback configurations for each group were different: (1) Group One -What's wrong, (2) Group Two -What's wrong + Gap, (3) Group Three -What's wrong + Gap + Hint. This study found that simply knowing what failed did not help students sufficiently, and might stimulate system gaming behavior. Hints were not found to be impactful on student performance or their usage of automated feedback. Based on the findings, this study provides practical guidance on the design of automated feedback.

show abstract

Section: B System Implementation and Feedback Designmentioning

confidence: 99%

Investigating the Essential of Meaningful Automated Formative Feedback for Programming Assignments

Hao

Wilson

Ottaway

et al. 2019

2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)

View full text Add to dashboard Cite

show abstract

“…These systems offer limited feedback to the student. Recently there has been some effort to provide better feedback using automatic grading [2,5,6,11]. In particular, in [6], the authors utilize existing autograding systems and designed knowledge maps to classify common errors to automatically generate hints meant to address said errors.…”

Section: Introductionmentioning

confidence: 99%

Effects of Human vs. Automatic Feedback on Students' Understanding of AI Concepts and Programming Style

Leite

Blanco

2020

Proceedings of the 51st ACM Technical Symposium on Computer Science Education

View full text Add to dashboard Cite

The use of automatic grading tools has become nearly ubiquitous in large undergraduate programming courses, and recent work has focused on improving the quality of automatically generated feedback. However, there is a relative lack of data directly comparing student outcomes when receiving computer-generated feedback and human-written feedback. This paper addresses this gap by splitting one 90-student class into two feedback groups and analyzing differences in the two cohorts' performance. The class is an intro to AI with programming HW assignments. One group of students received detailed computer-generated feedback on their programming assignments describing which parts of the algorithms' logic was missing; the other group additionally received human-written feedback describing how their programs' syntax relates to issues with their logic, and qualitative (style) recommendations for improving their code. Results on quizzes and exam questions suggest that human feedback helps students obtain a better conceptual understanding, but analyses found no difference between the groups' ability to collaborate on the final project. The course grade distribution revealed that students who received human-written feedback performed better overall; this effect was the most pronounced in the middle two quartiles of each group. These results suggest that feedback about the syntax-logic relation may be a primary mechanism by which human feedback improves student outcomes. CCS CONCEPTS • Social and professional topics → Computer science education; Computational thinking; Student assessment.

show abstract

“…In case of a compiler error, it provides both the erroneous line of the other student, as well as the modified line which resulted in successful compilation as a hint. GradeIT [17] uses simple rewrite rules to repair common compilation errors. Their study show that even these simple repairs can be effective for feedback generation and automated grading of assignments.…”

Section: Methodsmentioning

confidence: 99%

TipsC: Tips and Corrections for programming MOOCs

Sharma

Agarwal

Mor

2018

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

With the widespread adoption of MOOCs in academic institutions, it has become imperative to come up with better techniques to solve the tutoring and grading problems posed by programming courses. Programming being the new 'writing', it becomes a challenge to ensure that a large section of the society is exposed to programming. Due to the gradient in learning abilities of students, the course instructor must ensure that everyone can cope up with the material, and receive adequate help in completing assignments while learning along the way. We introduce TipsC for this task. By analyzing a large number of correct submissions, TipsC can search for correct codes resembling a given incorrect solution. Without revealing the actual code, TipsC then suggests changes in the incorrect code to help the student fix logical runtime errors. In addition, this also serves as a cluster visualization tool for the instructor, revealing different patterns in user submissions. We evaluated the effectiveness of TipsC's clustering algorithm on data collected from previous offerings of an introductory programming course conducted at IIT Kanpur where the grades were given by human TAs. The results show the weighted average variance of marks for clusters when similar submissions are grouped together is 47% less compared to the case when all programs are grouped together.

show abstract

Automatic Grading and Feedback using Program Repair for Introductory Programming Courses

Cited by 72 publications

References 13 publications

Investigating the Essential of Meaningful Automated Formative Feedback for Programming Assignments

Investigating the Essential of Meaningful Automated Formative Feedback for Programming Assignments

Effects of Human vs. Automatic Feedback on Students' Understanding of AI Concepts and Programming Style

TipsC: Tips and Corrections for programming MOOCs

Contact Info

Product

Resources

About