Rubric development and validation for assessing educational robotics skills

Abstract

The current study describes the process of development and validation of a rubric for assessing educational robotics skills among middle school students. A multiphase method has been followed, including a literature study, expert consultation, content validation, pilot testing, reliability analysis, and construct validation. The resulting dual-category rubric designing and programming skills were further elaborated by specific criteria and performance levels. Content validation identified seven key criteria: stability and durability, motors, innovation, code organization, dependability and reliability, movement and rotation, and innovative use of sensors. The results of inter-rater reliability analysis for all criteria indicated good agreement, with Krippendorff’s alpha ranging from 0.941 to 1.000. Additionally, construct validation using exploratory factor analysis confirmed a two-factor structure that aligns with the intended domains of design and programming skills, explaining 67.4% of the total variance. The rubric was consistent with learning/teaching theories such as Bloom’s taxonomy, constructivist learning theory, and self-determination theory. This rubric fulfills the existing need for a specific assessment tool in educational robotics and supports educational practices through very detailed feedback to students on their performance. This adds much to educational assessment and education in robotics and enhances teaching and learning outcomes for STEAM education in various settings.