Benchmarking Introductory Programming Exams

Simón,; Sheard, Judy; D'Souza, Daryl; Klemperer, Peter; Porter, Leo; Sorva, Juha; Stegeman, Martijn; Zingaro, Daniel

doi:10.1145/2899415.2899473

Cited by 6 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Detailed analyses of written exam papers from institutions in a number of countries explore the types of question used [608], the difficulty of the questions [610], and their complexity [595]. Questions suitable for benchmarking have been proposed and trialled in exams in multiple institutions by Sheard, Simon, and colleagues [597,612,613]. Another analysis of exam papers from multiple institutions by Petersen et al [506] finds that most questions focus on code writing and involve multiple concepts, making them challenging for novices.…”

Section: Assessment Analysismentioning

confidence: 99%

Introductory programming: a systematic literature review

Luxton-Reilly

Simón

Albluwi

et al. 2018

Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education

Self Cite

360

127

View full text Add to dashboard Cite

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming.This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research.

show abstract

Section: Assessment Analysismentioning

confidence: 99%

Introductory programming: a systematic literature review

Luxton-Reilly

Simón

Albluwi

et al. 2018

Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education

Self Cite

360

127

View full text Add to dashboard Cite

show abstract

“…It is clear to us that there is no single correct decomposition for a given question; indeed, the members of our expert group did not always agree on how a certain concept should be decomposed. For example, in question 6 from the benchmarking paper [75], the statement i n t [ ] nums1 = { 1 , −5 , 2 , 0 , 4 , 2 , −3 } ; led to disagreement on whether the concept of references is necessary as a token. A discussion revealed the idea that the granularity of decomposition is context-dependent, and that the concept of references is not needed in the decomposition of this particular code in the context of this particular question.…”

Section: Case Study Summarymentioning

confidence: 99%

“…The students scored very poorly -22.89 out of 110 possible points on average. As a result, the paper has been widely cited since its publication in 2001 for the proposition that students cannot write code (see, e.g., [24,29,38,75]).…”

Section: Introductionmentioning

confidence: 99%

Developing Assessments to Determine Mastery of Programming Fundamentals

Luxton-Reilly

Becker

Cao

et al. 2018

Proceedings of the 2017 ITiCSE Conference on Working Group Reports

View full text Add to dashboard Cite

Current CS1 learning outcomes are relatively general, specifying tasks such as designing, implementing, testing and debugging programs that use some fundamental programming constructs. These outcomes impact what we teach, our expectations, and our assessments. Although prior work has demonstrated the utility of single concept assessments, most assessments used in formal examinations combine numerous heterogeneous concepts, resulting in complex and difficult tasks. As a consequence, teachers may not be able to diagnose the actual difficulties faced by students and students are not provided with accurate feedback about their achievements. Such limitations on the nature and quality of feedback to teachers and students alike may contribute to the perceived difficulty and high dropout rates commonly observed in introductory programming courses.In this paper we review the concepts that CS education researchers have identified as important for novice programming. We survey learning outcomes for introductory programming courses that characterize the expectations of CS1 courses, and analyse assessments designed for CS1 to determine the individual components of syntax and semantics required to complete them. Having recognized the implicit and explicit expectations of novice programming courses, we look at the relationships between components and progression between concepts. Finally, we demonstrate how some complex assessments can be decomposed into atomic elements that can be assessed independently.Pre-print of the paper (accepted manuscript) for the institutional repository and not for redistribution. See terms of the ACM Copyright Transfer Agreement.

show abstract

Automatic Categorization of Introductory Programming Students

Rubio

2019

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

Benchmarking Introductory Programming Exams

Cited by 6 publications

References 9 publications

Introductory programming: a systematic literature review

Introductory programming: a systematic literature review

Developing Assessments to Determine Mastery of Programming Fundamentals

Automatic Categorization of Introductory Programming Students

Contact Info

Product

Resources

About