Stealth assessment: a theoretically grounded and psychometrically sound method to assess, support, and investigate learning in technology-rich environments

Rahimi, Seyedahmad; Shute, Valerie J.

doi:10.1007/s11423-023-10232-1

Cited by 5 publications

(2 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, this automated assessment could be replicated in other similar learning environments. When the automated assessment functions in real time, we will have a complete cycle of automated assessment in place as this assessment method is meant to be mostly used for formative rather than summative purposes (Rahimi & Shute, 2023).…”

Section: Limitations and Future Researchmentioning

confidence: 99%

“…The automated, unobtrusive assessment we discuss in this article is based on a formative assessment technique (i.e., assessment for learning; see Shute & Rahimi, 2017) called stealth assessment, introduced by Shute (2011). Stealth assessment uses digital learning environments (historically, digital games) as vehicles for assessing and supporting various competencies such as basic knowledge and skills including physics (Rahimi & Shute, 2023; Shute & Rahimi, 2021); and other hard-to-measure skills such as creativity (Rafner, 2021; Shute & Rahimi, 2021), persistence (DiCerbo, 2014; Rahimi et al, 2021, 2022), and problem-solving (Akram et al, 2018). This assessment method was originally called stealth because it is woven into the learning process and is invisible to the learners.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An automated, unobtrusive, formative assessment of creativity in a computer science and music remixing learning environment.

Rahimi,

Smith,

Truesdell

et al. 2024

Psychology of Aesthetics, Creativity, and the Arts

View full text Add to dashboard Cite

Creativity is one of the most crucial skills for success in the 21st-century workforce. Specifically, creativity is an important skill to have in science, technology, engineering, and mathematics (STEM)-related fields, and more empirical studies are needed to assess and improve creativity in STEM-related learning environments. In this study, we designed and validated an automated, unobtrusive, formative assessment of creativity in EarSketch, a computational music remixing platform where students learn to write Python or JavaScript code to create pieces of music. Using an existing data set of EarSketch projects (n = 53), we addressed two research questions: (Research Question 1) To what extent is the automated assessment of creativity that we designed in EarSketch psychometrically sound (focusing on validity and reliability), and (Research Question 2) what variables (i.e., divergent thinking, complexity, and self-report variables) predict students' creativity in EarSketch? Our main findings show that (a) the automated assessment of creativity has reasonable convergent validity (r = .47) and discriminant validity; (b) the automated assessment of creativity has a reliability estimate of .70; and (c) divergent thinking and the students' confidence in learning how to code significantly predicted students' creativity scores in an external, consensual assessment of creativity by EarSketch experts. Providing learning environments that can assess and support essential skills such as creativity alongside other STEM-related skills such as programming and computational thinking holds great promise for developing the next generation of the workforce who is not merely aware of STEM concepts and principles, but is creative and innovative in pursuing STEM solutions.

show abstract

Section: Limitations and Future Researchmentioning

confidence: 99%