Computational Thinking (CT ), entailing both domain-general and domain-specific skills, is a competency fundamental to computing education and beyond. However, as a cross-domain competency, appropriate assessment design and method remain equivocal. Indeed, the majority of the existing assessments have a predominant focus on measuring programming proficiency and neglecting other contexts in which CT can also be manifested. To broaden the promotion and practice of CT, it is necessary to integrate diverse problem types and item formats using a competency-based assessment method to measure CT. Taking a psychometric approach, this article evaluates a novel computer-based assessment of CT competency, Computational Thinking Challenge. The assessment was administered to 119 British upper secondary school students ( M = 16.11; SD = 1.19) with a range of prior programming experiences. Results from several reliability analyses, a convergent validity analysis, and a Rasch analysis, provided evidence to support the quality of the assessment. Taken together, the study demonstrated the feasibility to expand from traditional assessment methods to integrating multiple contexts, problem types, and item formats in measuring CT competency in a comprehensive manner.

Computational thinking (CT) is an emerging and multifaceted competence important to the computing era. However, despite the growing consensus that CT is a competence domain, its theoretical and empirical account remain scarce in the current literature. To address this issue, rigorous psychometric evaluation procedures were adopted to investigate the structure of CT competency, as measured by Computational Thinking Challenge (Lai, 2021a), in a large sample of 1,130 British secondary school students ( M age = 14.14 years, SD age = 1.45). Based on model comparison from an exploratory multidimensional item response theory approach, the results supported the multidimensional operationalization of CT competency. A confirmatory bi-factor item response theory model further suggested CT competency is comprised of a general CT competency factor and two specific factors for programming and non-programming problem-solving. Despite the multidimensionality, the common variance is largely explained by a primary general factor of CT competency, thus the use of a single scale score is recommended. Psychometric evaluation from the bi-factor model indicated good psychometric properties of the assessment tool. Overall, the bi-factor model provides a useful approach to investigating CT competency and serves as a robust test validation tool.

As a dynamic and multifaceted construct, computational thinking (CT) has proven to be challenging to conceptualize and assess, which impedes the development of a workable ontology framework. To address this issue, the current article describes a novel approach towards understanding the ontological aspects of CT by using text mining and graph-theoretic techniques to elucidate teachers’ perspectives collected in an online survey (N = 105). In particular, a hierarchical cluster analysis, a knowledge representation method, was applied to identify sub-groups in CT conceptualization and assessment amongst teachers. Five clusters in conceptualization and two clusters in assessment were identified; several relevant and distinct themes were also extracted. The results suggested that teachers attributed CT as a competence domain, relevant in the problem- solving context, as well as applicable and transferrable to various disciplines. The results also shed light on the importance of using multiple approaches to assess the diversity of CT. Overall, the findings collectively contributed to a comprehensive and multi-perspective representation of CT that refine both theory and practice. The methodology employed in this article has suggested a minor but significant step towards addressing the quintessential questions of “what is CT?” and “how is it evidenced?”.

Computational thinking (CT) is an emerging and multifaceted competence important tothe computing era. However, despite the growing consensus that CT is a competence domain, its theoretical and empirical account remain scarce in the current literature. To address this issue, rigorous psychometric evaluation procedures were adopted to investigate the structure of CT competency, as measured by Computational Thinking Challenge (Lai, 2021b), in a large sample of 1,130 British secondary school students (Mage = 14.14 years, SDage = 1.45). Based on model comparison from an exploratory multidimensional item response theoryapproach, the results supported the multidimensional operationalization of CT competency. A confirmatory bi-factor item response theory model further suggested CT competency is comprised of a general CT competency factor and two specific factors for programming and non-programming problem-solving. Despite the multidimensionality, the common variance is largely explained by a primary general factor of CT competency, thus the use of a single scale score is recommended. Psychometric evaluation from the bi-factor model indicated good psychometric properties of the assessment tool. Overall, the bi-factor model provides a useful approach to investigating CT competency and serves as a robust test validation tool.

No abstract