Where exactly are the difficulties in reasoning logically about code? experimentation with an online system

Tool-Aided Learning of Code Reasoning with Abstraction in the CS Curriculum

“…It is just not possible to run every possible test scenario. For example, in a junior level software engineering course, 50% of students found the code segment in Listing 1 was a correct implementation for finding the maximum value between two integers (Cook et al, 2018).…”

“…A key aim of using automated tools is to understand the specific fine-grain learning difficulties students face when reasoning about code so that appropriate interventions may be developed (Cook et al, 2018;Fowler et al, 2019;Priester et al, 2016). By "fine-grain," we mean understanding student difficulties at a resolution that exceeds identifying high-level constructs that might present challenges (e.g., functions, loops, parameter passing) to reveal the underlying cause(s) of a learning roadblock (e.g., a missing algebraic foundation or a flaw in the student's mental model of variable storage).…”

“…When a student's answer is wrong (i.e., does not verify, or is trivial), the lesson may be repeated, or a follow-on lesson is given. Though not a focus of the current paper, a key benefit of this tool is that it can pinpoint obstacles specific to subsets of learners (Cook et al, 2018).…”

“…Basics Given the focus on assertions, the code in each lesson is relatively simple. After a short introduction, students have little to no difficulty understanding the code, though the syntax is slightly different than what they are used to (e.g., the distinguished argument is passed as an explicit parameter -Push(K, S), instead of s.push(k)) (Cook et al, 2018).…”

“…Symbolic reasoning basics concerns the ability to reason about a sequence of assignment statements, and that is the focus of this section. Additional details of symbolic reasoning studies in pinpointing specific difficulties for individual students and subgroups may be found in (Cook et al, 2018). A related study in (Fowler et al, 2019) explores the role of motivation in learning symbolic reasoning.…”