Review of Programming Assignments Automated Assessment Systems

Tarek, Mohamed; Ashraf, Ali; Heidar, Mahmoud; Eliwa, Essam

doi:10.1109/miucc55081.2022.9781736

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2023

2024

Publication Types

Select...

Article2

Book1

Other1

Relationship

Self Cite0

Independent4

Authors

Journals

Cited by 4 publications

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Generation of Student’s Programming Exercises Using SCT Generator

Vusić,

Radošević,

Bernik

2024

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

Generation of Student’s Programming Exercises Using SCT Generator

Vusić,

Radošević,

Bernik

2024

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

Evaluation of LLM Tools for Feedback Generation in a Course on Concurrent Programming

Estévez-Ayres,

Callejo,

Hombrados-Herrera

et al. 2024

Int J Artif Intell Educ

View full text Add to dashboard Cite

The emergence of Large Language Models (LLMs) has marked a significant change in education. The appearance of these LLMs and their associated chatbots has yielded several advantages for both students and educators, including their use as teaching assistants for content creation or summarisation. This paper aims to evaluate the capacity of LLMs chatbots to provide feedback on student exercises in a university programming course. The complexity of the programming topic in this study (concurrency) makes the need for feedback to students even more important. The authors conducted an assessment of exercises submitted by students. Then, ChatGPT (from OpenAI) and Bard (from Google) were employed to evaluate each exercise, looking for typical concurrency errors, such as starvation, deadlocks, or race conditions. Compared to the ground-truth evaluations performed by expert teachers, it is possible to conclude that none of these two tools can accurately assess the exercises despite the generally positive reception of LLMs within the educational sector. All attempts result in an accuracy rate of 50%, meaning that both tools have limitations in their ability to evaluate these particular exercises effectively, specifically finding typical concurrency errors.

show abstract

AI chatbots in programming education: guiding success or encouraging plagiarism

Akçapınar,

Sidan

2024

Discov Artif Intell

View full text Add to dashboard Cite

Review of Programming Assignments Automated Assessment Systems

Cited by 4 publications

References 25 publications

Generation of Student’s Programming Exercises Using SCT Generator

Generation of Student’s Programming Exercises Using SCT Generator

Evaluation of LLM Tools for Feedback Generation in a Course on Concurrent Programming

AI chatbots in programming education: guiding success or encouraging plagiarism

Contact Info

Product

Resources

About