Towards a Common Instrument for Measuring Prior Programming Knowledge

Abstract

Computing education researchers and educators use a wide range of approaches for measuring students' prior knowledge in programming. Such measurement can help adapt the learning goals and assessment tools for groups of learners at different skills levels and backgrounds. There seems to be no consensus on if and how prior programming knowledge should be measured. Traditional background surveys are often ad-hoc or non-standard, which do not allow comparison of results between different course contexts, levels, and learner groups. Moreover, surveys may yield inaccurate information and may not be useful due to lack of detail. In contrast, tests can provide much higher detail and accuracy than surveys about student knowledge or skills, but large-scale tests are typically very time-consuming or impractical to arrange. To bridge the gap between ad-hoc surveys and standardized tests, we propose and evaluate a novel self-evaluation instrument for measuring prior programming knowledge in introductory programming courses. This instrument investigates in higher detail typical course concepts in programming education considering the different levels of proficiency. Based on a sample of two thousand introductory programming course students, our analysis shows that the instrument is internally consistent, correlates with traditional background information metrics and identifies students of varying programming backgrounds. CCS CONCEPTS• Social and professional topics → Computer science education; Model curricula; Student assessment.

Exploring the Value of Student Self-Evaluation in Introductory Programming

“…This article continues the work of Duran et al [13] to develop and validate a self-evaluation instrument (SEI). The SEI complements existing assessment tools by rapidly surveying students' understanding of introductory programming concepts.…”

“…The next section summarizes the development and structure of our SEI, as originally presented by Duran et al [13]. The subsequent sections present our study design and answer the five research questions listed above.…”

“…Our prior work [13] explored the metrics of internal consistency in a pilot by analyzing the prototype SEI answers of 2196 students at the beginning of week 1. These results indicated that the instrument is highly reliable (Cronbach's α = .98) and could distinguish itself from traditional metrics [15], while strongly correlating with them.…”

“…The course is taken by both degree and non-affiliated students at the University of Helsinki. When starting the course, students fill in a research consent form and background questionnaire (Duran et al [13] explores the relationship between the SEI and background questions in more detail). Answering the questionnaire was voluntary but included an incentive raffle of movie tickets.…”

Quantifying the Effects of Programming Education on Students’ Knowledge Representation and Perception in Computational Thinking

“…There are various ways to represent and measure knowledge. In computer science education, a study was conducted on tools that define the concept and form of programming knowledge and thought expression (Gürsoy, 2016;Duran, 2019). In addition to the validity of the questionnaire, major analyses were performed and correlations between concepts were analysed.…”