Computational Thinking Assessment – Towards More Vivid Interpretations

Abstract

Computational thinking (CT) is an important 21st-century skill. This paper aims at more useful CT assessment. Available evaluation instruments are reviewed; two generally accepted CT evaluation tools are selected for a comprehensive CT assessment: the CTt, a performance test, and the CTS, a self-assessment instrument. The sample comprises 202 high school students from German-speaking Switzerland. Concerning the CTt, Raschscalability is demonstrated. Utilizing the approach of the PISA studies, proficiency levels are formed that comprise tasks with specific characteristics that students are systematically able to master. This could help teachers to offer individual support to their students. In terms of the CTS, the original version is refined using confirmatory factor and measurement-invariance analysis. A latent profile analysis yielded four profiles, two of which are of particular interest. One profile comprises students with, on the one hand, moderate to high creative thinking ability, cooperativity, and critical thinking skills and, on the other hand, low algorithmic thinking ability. The second remarkable profile consists of students with particularly low cooperativity. Based on these strength and weakness profiles, teachers could offer support tailored to student needs.

Comparing the psychometric properties of two primary school Computational Thinking (CT) assessments for grades 3 and 4: The Beginners' CT test (BCTt) and the competent CT test (cCTt)

“…Future work should therefore consider continuing to refine the limits of validity of the instruments. Indeed, refinement studies are common in educational psychology, with similar work having already been undertaken for (i) the original CTt (aimed at 10–16 year old students) to improve it's validity for 16 year old students and above (Guggemos et al, 2022 ), and (ii) The TechCheck and it's variants to improve the validity for kindergarden students (Relkin et al, 2020 ; Relkin and Bers, 2021 ).…”

competent Computational Thinking Test

“…Indeed, for researchers, it would be possible to determine how an intervention affects individuals or groups of individuals over extended periods of time, therefore providing more reliable insight into the relevance of an intervention beyond short term interventions which are presently the most common in the field. For educators on the other hand, such assessments may help establish student profiles and help target their classroom interventions and offer tailored support that is adapted to students' needs (Guggemos et al, 2022). Finally, for practitioners, to evaluate the longitudinal impact of widespread computing-related curricular reforms, it is essential to have validated assessments throughout K-12 to follow students' progress over time.…”

“…Establishing student proficiency profiles. As "an assessment is not an end in itself" and given that "it should contribute to promoting student learning (Pellegrino et al, 2016)" (Guggemos et al, 2022), proficiency profiles are established through IRT in order to improve the utility of the cCTt for researchers, educators and practitioners. The objective is to provide profiles "ranging from very low levels of proficiency to very high levels of proficiency", which each "describ [ing] what students typically know and can do at [said]] level of proficiency" (OECD, 2017).…”

“…The objective is to provide profiles "ranging from very low levels of proficiency to very high levels of proficiency", which each "describ [ing] what students typically know and can do at [said]] level of proficiency" (OECD, 2017). These profiles are established as was done by Guggemos et al (2022) for the CTt, by drawing inspiration from the approach employed by the OECD for the PISA assessments (OECD, 2014(OECD, , 2017 and the fact that a student of a given ability "are increasingly more likely to complete tasks located at progressively lower points on the scale, and increasingly less likely to complete tasks located at progressively higher points on the scale" (OECD, 2017). Therefore, based on the outputs of IRT models, multiple proficiency levels are constructed with respect to the test's item difficulties on the logit scale.…”

“…From the practitioners' perspective, assessment issues need to be resolved for successful integration of CT in K-12 curricula (Cutumisu et al, 2019). This is because the "purpose of an assessment is to facilitate student learning" (Guggemos et al, 2022) and "validated assessments [...] measure students' progress in meeting the learning outcomes prescribed by the programs of study" (Cutumisu et al, 2019). It thus becomes paramount to develop and guide "researchers and practitioners in choosing developmentally appropriate CT assessments" (Cutumisu et al, 2019).…”