Static analysis of programming exercises: Fairness, usefulness and a method for application

Nutbrown, Stephen; Higgins, Colin

doi:10.1080/08993408.2016.1179865

Cited by 17 publications

(14 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We advise educators to customize the tool by selecting a small set of relevant checks and adjusting threshold values. Other recommendations for using PMD for assessing programming exercises have been proposed by Nutbrown and Higgins [9].…”

Section: Discussionmentioning

confidence: 99%

Code Quality Issues in Student Programs

Keuning

Heeren

Jeuring

2017

Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education

View full text Add to dashboard Cite

Because low quality code can cause serious problems in software systems, students learning to program should pay attention to code quality early. Although many studies have investigated mistakes that students make during programming, we do not know much about the quality of their code. This study examines the presence of quality issues related to program ow, choice of programming constructs and functions, clarity of expressions, decomposition and modularization in a large set of student Java programs. We investigated which issues occur most frequently, if students are able to solve these issues over time and if the use of code analysis tools has an e ect on issue occurrence. We found that students hardly x issues, in particular issues related to modularization, and that the use of tooling does not have much e ect on the occurrence of issues.

show abstract

Section: Discussionmentioning

confidence: 99%

Code Quality Issues in Student Programs

Keuning

Heeren

Jeuring

2017

Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education

View full text Add to dashboard Cite

show abstract

“…[13] state that a small sample size (5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20) is sufficient for usability assessment and [7] highlights that researchers can collect valuable results using a minimum of five participants (subject experts) in a main experiment. Furthermore, [20] used five subject experts in order to provide feedback for programming solutions in their main experiment. Given the difficulties associated with finding subject expert willing to FIGURE 3 Overview of the partial marking model spend sufficient time to run the experiments the sample size is fairly low.…”

Section: Participants and Questionsmentioning

confidence: 99%

Improving marking efficiency for longer programming solutions based on a semi‐automated assessment approach

Buyrukoğlu

Batmaz

Lock

2019

Comp Applic In Engineering

View full text Add to dashboard Cite

In recent years, many students in higher education have begun to learn programming languages. In doing so they will complete a variety of programming tasks of varying degrees of complexity. The students need to get consistent and personalized feedback to develop their programming skills. Human markers can provide personalized feedback using traditional manual approaches to assessment, but they may provide inconsistent feedback (especially for long programming solutions) since marking the programming solutions of multiple students can represent a significant workload for them. While full‐automated assessment systems are the best to provide consistent feedback, they may not provide sufficiently personalized feedback for novice programmers. This study develops a novel semi‐automated assessment approach in order to improve efficiency of human marker in the marking process and increase consistency of feedback (for both short and long programming solutions). It advocates the reuse of human marker's comments for similar code snippets, defined as segmented marking in this study. New full and partial marking models are developed based on segmented marking and they are tested by expert markers. The findings show that the two models are similar in efficiency, but that a partial marking approach potentially offers an improved efficiency for longer programming solutions. Such a finding has significant potential to reduce time spent on marking throughout the sector, which would have significant impact on both resourcing and timeliness of feedback.

show abstract

“…The Marker's Apprentice (TMA) [19], building upon CourseMarker [20], explores the use of rule-based codecorrectness metrics to automatically determine the grade to be awarded to a submission. While some of these metrics are absolute (such as the presence of empty code blocks or incorrectly testing for equality), others require interpretation (such as determining the acceptable range of cyclomatic complexity or the total number of classes submitted).…”

Section: Metricsmentioning

confidence: 99%

“…While this use of metrics has been shown to be useful, it is not error-proof therefore using it in isolation is not advised. Instead, the use of metrics should be used as part of a suite of tools with a human assessor to provide some oversight and custom feedback comments, where possible [19].…”

Section: Metricsmentioning

confidence: 99%

“…For example, while eCAF [15] and other comment-banks can enable providing feedback to a task of any size, it is primarily human driven to write the comment and attach it to a submission. Tools such as TMA [19], the unnamed tool by Piech [23], and OverCode [22] can enable the application of comments to multiple submissions, their limitations include a significant up-front cost to pre-empt the range of possible solutions and to create tests/define metrics ranges specific to that task. Other tools focus upon the automated grading of programming task submissions.…”

Section: E Summarymentioning

confidence: 99%

See 1 more Smart Citation

Making the Most of Repetitive Mistakes: An Investigation into Heuristics for Selecting and Applying Feedback to Programming Coursework

Howell

Wong

2018

2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE)

View full text Add to dashboard Cite

In the acquisition of software-development skills, feedback that pinpoints errors and explains means of improvement is important in achieving a good student learning experience. However, it is not feasible to manually provide timely, consistent, and helpful feedback for large or complex coursework tasks, and/or to large cohorts of students. While tools exist to provide feedback to student submissions, their automation is typically limited to reporting either test pass or failure or generating feedback to very simple programming tasks. Anecdotal experience indicates that clusters of students tend to make similar mistakes and/or successes within their coursework. Do feedback comments applied to students' work support this claim and, if so, to what extent is this the case? How might this be exploited to improve the assessment process and the quality of feedback given to students? To help answer these questions, we have examined feedback given to coursework submissions to a UK level 5, university-level, data structures and algorithms course to determine heuristics used to trigger particular feedback comments that are common between submissions and cohorts. This paper reports our results and discusses how the identified heuristics may be used to promote timeliness and consistency of feedback without jeopardising the quality.

show abstract

Static analysis of programming exercises: Fairness, usefulness and a method for application

Cited by 17 publications

References 8 publications

Code Quality Issues in Student Programs

Code Quality Issues in Student Programs

Improving marking efficiency for longer programming solutions based on a semi‐automated assessment approach

Making the Most of Repetitive Mistakes: An Investigation into Heuristics for Selecting and Applying Feedback to Programming Coursework

Contact Info

Product

Resources

About