Using Dr. Scratch as a Formative Feedback Tool to Assess Computational Thinking

Abstract

This study investigated if using Dr. Scratch as a formative feedback tool would accelerate students' Computational Thinking (CT). Forty-one 4th-6th grade students participated in a 1-hour/week Scratch workshop for nine weeks. We measured pre- and posttest results of the computational thinking test (CTt) between control (n = 18) and treatment groups (n = 23) using three methods: propensity score matching (treatment = .575; control = .607; p = .696), information maximum likelihood technique (treatment effect = -.09; p = .006), and multiple linear regression. Both groups demonstrated significantly increased posttest scores over their pretest (treatment = +8.31%; control = +5.43%), showing that learning to code can increase computational thinking over a 2-month period. In this chapter, we discuss the implications of using Dr. Scratch as a formative feedback tool the possibilities of further research on the use of automatic feedback tools in teaching elementary computational thinking.

Analyzing computational thinking studies in Scratch programming: A review of elementary education literature

“…In addition to identifying these traits, Dr. Scratch then extrapolates and assigns a CT score to projects, thus providing feedback to both educators and learners about the CT skills present in their work (Moreno-León et al, 2015, Moreno-León et al, 2017. What makes Dr. Scratch a powerful tool, however, is the general consensus on its effectiveness (Browning, 2017;Lawanto, 2016;Oluk & Korkmaz, 2016).…”

“…Lawanto (2016), for example, concluded that Dr. Scratch was well suited to assess computational skills, which in turn helped teachers understand students' strengths and weaknesses in programming. Browning (2017) conducted a pre/posttest study in which two groups of 5th-6th graders had a treatment group assessed by Dr. Scratch for the presence of CT skills, and where the control group was assessed by other CT tests. They found that the development in students' programming skills in Scratch would relate to similar increases in their computational thinking skills or improvements in their computational thinking levels.…”

“…They found that the development in students' programming skills in Scratch would relate to similar increases in their computational thinking skills or improvements in their computational thinking levels. Nevertheless, Dr. Scratch, as a tool, is not without its own limitations which both Lawanto (2016) and Browning (2017) noted, chiefly in the area of formative assessment. That is, Dr. Scratch is a summative assessment by nature and not one that provides feedback during the learning process.…”

“…This was especially noticeable for students with an initial medium (i.e., developing) CT score rather than for students with a high one. Browning (2017), however, took a somewhat different approach to utilizing Dr. Scratch for formative assessment purposes. The difference was in taking certain components of programming into account (i.e., easier versus more difficult tasks or skills).…”

Effect of SRA-programming on computational thinking through different output modalities

“…It encompasses the mental skills and practices needed to design computations that can let computers perform tasks for us, and to explain and interpret the world as a complex system of information processes (Denning & Tedre, 2019;Lai, 2021). The provision of opportunities to further stimulate the development of CT, and to enable pupils to acquire the associated skills, entails demands on both the environment and the task (Kong & Abelson, 2019;Rich & Browning, 2022;Yadav et al, 2016).…”