Development and Validation of Computational Thinking Assessment of Chinese Elementary School Students

Abstract

Recent advances in artificial intelligence have brought attention to computational thinking (CT) in school education worldwide. However, little is known about the development of the literacy of CT in children, mainly because of the lack of proper psychometric assessments. We developed the first psychometrically validated assessment on the literacy of CT of children in Chinese elementary schools, coined as the Computational Thinking Assessment for Chinese Elementary Students (CTA-CES). Items were constructed to reflect key aspects of CT such as abstraction, algorithm thinking, decomposition, evaluation, and pattern recognition. To examine the test reliability and validity, we recruited two samples of 280 third- to sixth-grade students in total. Cronbach’s alpha provided evidence for the reliability of the test scores, item response theory analyses demonstrated psychometric appropriateness, whereas construct validity was verified by convergent validity, and criterion-related validity was confirmed by correlations between the CTA-CES and measures related to CT, namely reasoning, spatial ability, and verbal ability. In addition, an fMRI study further demonstrated similar neural activation patterns when students conducted the CTA-CES and programming tasks. Taken together, the CTA-CES is the first reliable and valid instrument for measuring the literacy of CT for Chinese children, and may be applicable to children worldwide.

Comparing the psychometric properties of two primary school Computational Thinking (CT) assessments for grades 3 and 4: The Beginners' CT test (BCTt) and the competent CT test (cCTt)

“…As numerous researchers have put forward, instruments such as the BCTt and the cCTt should be combined with other forms of assessments in a systems of assessments (Grover et al, 2015 ; Román-González et al, 2019 ; Weintrop et al, 2021a ) to accurately measure the full range of competencies at play when considering CT (Brennan and Resnick, 2012 ; Piatti et al, 2022 ). The systems of assessments could therefore include other instruments which assess CT practices such as the test by Li et al ( 2021 ), employ direct observations of students' thought processes and strategies (Lye and Koh, 2014 ; Chevalier et al, 2020 ), or learning analytics and educational data mining techniques (Cock et al, 2021 ; Nasir et al, 2021 ; Zapata-Cáceres and Martín-Barroso, 2021 ). Complementary assessments would not only help gain a more accurate and in-depth picture of student learning but also feed into the learning activity design and intervention process (Chevalier et al, 2022 ).…”

“…The first is comparing with other existing validated assessments. For instance, Relkin et al ( 2020 ) compared the TechCheck with the TACTIC-KIBO, while (Li et al, 2021 ) went one step further and correlated the CTA-CES with reasoning, spatial abilities, and verbal abilities. The second is establishing the test's predictive validity, for example by establishing whether the instrument can predict academic performance and coding achievement as done by Román-González et al ( 2018 ).…”

“…Instruments meeting the aforementioned criteria, and having undergone a psychometric validation and reliability assessment process at the level of primary school (see section 2.2), include the TechCheck for lower primary school (grades 1–2, ages 6–8, Relkin et al, 2020 ), the TechCheck-K, which is an adaptation of the former for kindergarden (ages 4–6, Relkin and Bers, 2021 ), the BCTt for grades 1–6 (ages 5–10, Zapata-Cáceres et al, 2020 ), the competent CT test (cCTt) for grades 3–4 (ages 7–9, El-Hamamsy et al, 2022a ), the Computational Thinking Assessment for Chinese Elementary Students (CTA-CES) for grades 3–6 (ages 9–12, Li et al, 2021 ), and Kong and Lai ( 2022 )'s CT-concepts test for grades 3–5. A synthesis of these instruments is provided in Table 1 and shows that these instruments often differ in the underlying definition of CT employed to define the test items which makes it complex to compare them pyschometrically.…”

Predisposition of Schoolchildren to Computational Thinking When Studying Digital Technologies

“…Разработана первая психометрически подтвержденная методика оценки грамотности детей в начальных школах Китая в области ВМ «Оценка вычислительного мышления для китайских учеников начальной школы» (CTA-CES). Ее пункты отражают ключевые аспекты ВМ: абстракция, алгоритмическое мышление, декомпозиция, оценка и распознавание образов [16]. Методы оценки ВМ школьников K-12 по 26 показателям были проанализированы в обзоре литературы, закодированы в соответствии с конкретными измеренными компетенциями и организованы в 3D-гибридную структуру шкалы ВМ [17].…”

AI in learning

“…The second article, "Development and validation of computational thinking assessment of Chinese elementary school students" (Li, Y. et al, 2021), is related to computational thinking (CT). Its definition, teaching, and evaluation have been discussed by various scholars (e.g., Grover & Pea, 2018;Hsu et al, 2018;Nouri et al, 2019).…”