Covariational reasoning and item context affect language in undergraduate mass balance written explanations

Abstract

Mass balance (MB) reasoning offers a rich topic for examination of students' scientific thinking and skills, as it requires students to account for multiple inputs and outputs within a system and apply covariational reasoning. Using previously validated constructed response prompts for MB, we examined 1,920 student constructed responses (CRs) aligned to an emerging learning progression to determine how student language changes from low (1) to high (4) covariational reasoning Levels. As students' abilities and thinking may change with Context, we used the same general prompt in six physiological contexts. We asked how Level and Context affect student language and what language is conserved across all Contexts at higher reasoning Levels. Using diversity methods, we found student language becomes more similar as covariational reasoning level increases. Using text analysis, we found context-dependent words at each Level; however, the type of context words changed. Specifically, at Level 1, students used context words that are tangential to MB reasoning, while Level 4 responses used words that specify inputs and outputs for the given item context. Further at Level 4, students shared 30% of language across the six contexts and leveraged context-independent words including rate, equal, and some form of slower / lower / smaller. Together, these data demonstrate that Context affects undergraduate MB language at all covariational reasoning levels, but that the language becomes more specific and similar as Level increases. These findings encourage instructors to foster context-independent, comparative, and summative language during instruction to functionally build MB and covariational reasoning skills across contexts.

Extending a Pretrained Language Model (BERT) using an Ontological Perspective to Classify Students’ Scientific Expertise Level from Written Responses

“…A key challenge of automatic evaluation of CR assessments in science remains that as students gain knowledge in the discipline, their use of domain speci c terminology can increase (Jescovitch et al, 2021). Further, detecting student reasoning in CR when explaining phenomenon can be complicated by the domain speci c terms based on the provided context of the item itself (Shiroda et al, 2023). This study anticipates contributing to STEM disciplinary education more broadly and, speci cally, to environmental science education by introducing a faster and more resource-e cient scoring algorithmic model from an ontological perspective.…”