Analyzing rich qualitative data to study pencil-puzzle-based assignments in CS1 and CS2

Abstract

Pencil puzzles (puzzles such as sudoku and many others that are designed to be solved by humans, promoting computational thinking) provide a natural context for CS1/2 assignments. In a prior work we analyzed Likert-scaled student responses and assignment/course grades to show that not only are such assignments effective but are also largely independent of gender and prior computing experience. This paper focuses on open-ended student comments, both to see if they provide additional insights about the assignments and student perceptions not apparent from the Likert-scaled responses, and to see if these comments are consistent with the results from the prior work. We surveyed over 1000 students who had used pencilpuzzle-based assignments and invited them to make open-ended comments in their survey responses. We used grounded theory to develop codes for the large volume of student survey comments, as well as for semi-structured interviews with the instructors and focus groups with student TAs. Statistical analysis of the coded comments identified several interesting relationships, such as students being appreciative of their learning even when they perceived the assignments as difficult, which were not available from the Likertscaled data. The analysis also confirmed that these assignments are largely gender-and experience-neutral. We conclude by discussing how these results and the coding process lead to improvements in assignment development and inform future research directions. CCS CONCEPTS• Social and professional topics → Computer science education; Computational thinking;