Exploring the Value of Student Self-Evaluation in Introductory Programming

Abstract

Programming teachers have a strong need for easy-to-use instruments that provide reliable and pedagogically useful insights into student learning. Currently, no validated tools exist for rapidly assessing student understanding of basic programming knowledge. Concept inventories and the SCS1 questionnaire can offer great benefits; this article explores the additional value that may be gained from relatively simple self-evaluation metrics. We apply a lightweight self-evaluation instrument (SEI) in an introductory programming course and compare the results to existing performance measures, such as examination grades and the SCS1. We find that the SEI has a similar correlation with a program-writing examination as the SCS1 does, although both instruments correlate only moderately with the examination and each other. Furthermore, students are much more likely to voluntarily answer the lightweight SEI than SCS1. Overall, our results suggest that both the SEI and other instruments need to be greatly improved and outline future work towards that end. CCS CONCEPTS• Social and professional topics → Computer science education; Model curricula; Student assessment.

Theories and Models of Emotions, Attitudes, and Self-Efficacy in the Context of Programming Education

“…In a few cases, the results of a new study were contrasted with those using a different instrument (code A5). For example, Duran et al [30] compared their own findings on comprehending programming concepts with those of Danielsiek et al [24], who explored students' algorithm skills and pseudocode writing and tracing.…”

Uses, Revisions, and the Future of Validated Assessments in Computing Education: A Case Study of the FCS1 and SCS1

“…In a study of 79 novice programmers (defined as having had at most non-Python programming course and never learned or used Python), findings supported the claim that more agency and information can lead to higher student motivation, but not necessarily improved student learning [75]. Java: [17] Python: [37] (mSCS1), [58] MATLAB: [2, 3] (MCS1) German: [66,67] Finnish: [15] With the rise of block-based programming environments, questions have been raised about how block-based programming and block-based programming with text-based programming (dualmodality programming) compare to text-based programming. For a study comparing dual-modality programming environments and instruction with text-based approaches to instruction, participant learning was measured via the SCS1 assessment and course exams for 673 undergraduates in a semester-long CS1 course [5].…”

“…These factors can make the assessment inaccessible to many students and researchers. In an effort to create a more lightweight assessment that is less obtrusive, Duran et al created the SEI (self-evaluation instrument) to rapidly assess student understanding of basic programming knowledge [15]. Duran et al used the SCS1 assessment, after translating it into Finnish, to create an argument for validity for the SEI.…”

“…Timmerman et al then used the German translation of the SCS1 to monitor student learning in a CS1 course as the course was revised. A team in Finland followed Timmerman et al 's steps and translated the SCS1 into Finnish [15]. This was then used to create the SEI, described in Section 3.1.2.…”

Narrowing and Stretching: Addressing the Challenge of Multi-track Programming

“…Students were advised that the Black route was for those with Alevel Computer Science or equivalent experience, and were initially allocated on that basis. A lightweight web-based self-evaluation questionnaire [12] was administered in the first week and the distribution of results was shared with students so that they could identify their experience level compared with the rest of the cohort. Students were given the opportunity to change between the two modules without constraint during the first three weeks of teaching.…”