Automated Assessment of Computer Programming Practices: The 8-Years UNED Experience

Galán, Daniel; Heradio, Rubén; Vargas, Héctor; Abad, Ismael; Cerrada, J.A.

doi:10.1109/access.2019.2938391

Cited by 28 publications

(23 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are numerous related works in the literature on AA‐PE [1,8,9,23,23,29]. However, this section focuses on studies that are most relevant and related to the proposal in this paper.…”

Section: Background On Automated Assessment Of Programming Questionsmentioning

confidence: 99%

An experience of automated assessment in a large‐scale introduction programming course

Zampirolli

Josko

Venero

et al. 2021

Comp Applic In Engineering

View full text Add to dashboard Cite

The 2020 pandemic imposed new demands on teaching practices to support student's distance learning process. In this context, automated assessment (AA) is a pivotal resource that offers immediate and automatic feedback on students' programming tasks. Although the literature provides several contributions regarding AA of Programming Exercises (PEs), very few works discuss the automatic generation of personalized PE. This study reports our experience in applying a new proposal for AA‐PE in an Introduction to Programming (IP) course for a large group of students. This proposal's key feature is the ability to apply AA‐PE and parameterized unified exams to different programming languages by using the open‐source tools MCTest and Moodle (with virtual programming lab [VPL] plugin). During the first quarter of 2019, teachers of 19 of 44 IP‐FF (face‐to‐face) classes embraced our approach as a component in their pedagogical intervention. These classes achieved a higher pass rate (67.5%) than those that did not adopt our AA solution (59.1%), whereas the standard deviation was quite the same (22.5% and 21.3%, respectively). Additionally, preliminary results revealed a strong linear correlation (r = .93) between the pass rate and the average grade of the AA‐PE. In IP‐BL (blended learning), two classes used our method in the exams, with 171 students and a pass rate of 70.4%. These results corroborate previous works that continuous assessment combined with immediate feedback can contribute to students' learning process.

show abstract

“…There are numerous related works in the literature on AA‐PE [1,8,9,23,23,29]. However, this section focuses on studies that are most relevant and related to the proposal in this paper.…”

Section: Background On Automated Assessment Of Programming Questionsmentioning

confidence: 99%

An experience of automated assessment in a large‐scale introduction programming course

Zampirolli

Josko

Venero

et al. 2021

Comp Applic In Engineering

View full text Add to dashboard Cite

show abstract

“…Current paradigms for assessing students' competencies and skills in programming require tools capable of continuous assessment learning (Galan et al, 2019). In this sense, the authors presented an extensive study in a practical programming course from 2011 to 2018, in the Bachelor of Computer Science course, at the National University of Distance Education.…”

Section: Introductionmentioning

confidence: 99%

Parameterized and automated assessment on an introductory programming course

Zampirolli¹,

Pisani²,

Josko³

et al. 2020

Anais Do XXXI Simpósio Brasileiro De Informática Na Educação (SBIE 2020)

View full text Add to dashboard Cite

The generation of individualized exams can contribute to a more reliable assessment of the students. Manually performing this procedure may not be feasible, even more on a large scale. An alternative to deal with it is the automatic generation of questions. This paper discusses an innovative solution to simplify test generation and correction through parameterized questions in the context of a four-month Introduction to Programming course under a blended- learning (IP-BL) approach. It combines the open-source tool MCTest with Moodle and VPL plugin to generate and also automatically evaluate parameterized programming language questions. We applied an intervention based on this solution in two IP-BL groups (a total of 171 enrolled students) using Java.

show abstract

“…For this reason, computer programming MOOCs usually provide some kind of automatic evaluation system. To this purpose, MOOC systems use automatic graders to automatically assess all the programs (usually thousands of them) submitted by the students [25,28,34,41,42,46,46]. The automatic grader system is a key element of any MOOC framework.…”

Section: Introductionmentioning

confidence: 99%

“…Among these factors, we find the definition of complete test suites to check the output values as a common practice. Many AA tools are based on this criterion [14,16,17,25,25,28,40,52].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Semiautomatic generation and assessment of Java exercises in engineering education

Insa

Pérez

Silva

et al. 2020

Comp Applic In Engineering

View full text Add to dashboard Cite

Continuous assessment is essential in education. It should be an integral part of education that provides immediate feedback to students. Unfortunately, the assessment of programming source code is still a hand‐operated and error‐prone task, and can take weeks before the student gets feedback. This study presents a semiautomatic code assessment method able to automatically apply black‐box assessment (which relies on the comparison of input‐output pairs) and white‐box assessment (which relies on checking different source code properties). The method proposed is a general‐purpose assessment system that was originally designated to be used in engineering education, but that can be used in other educational contexts to assist the assessment of any Java programming assignments or exams. The main advantage of this system is that the assessment made is quicker, exhaustive, and objective; and it does not produce false positives. After the application of this method over two years in several real university courses, we have released a public and free implementation. An empirical evaluation of this system estimates that the amount of assessment work automatically done by the tool is over 48%. Additionally, the system has been used to measure the average subjective influence (i.e., assessment errors) introduced by teachers when they assess exams manually.

show abstract

Automated Assessment of Computer Programming Practices: The 8-Years UNED Experience

Cited by 28 publications

References 40 publications

An experience of automated assessment in a large‐scale introduction programming course

An experience of automated assessment in a large‐scale introduction programming course

Parameterized and automated assessment on an introductory programming course

Semiautomatic generation and assessment of Java exercises in engineering education

Contact Info

Product

Resources

About