Assessing high‐school students' modeling performance on Newtonian mechanics

Zhai, Xiaoming

doi:10.1002/tea.21758

Cited by 9 publications

(11 citation statements)

References 100 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hestenes (1992, p. 732) argues that "the great game of science is modeling the real world." Engaging students in developing and using models is considered a critical practice to improve students' scientific competence (Ke & Schwarz, 2021;Zhai, 2022;zu Belzen et al, 2019).…”

Section: An Inclusive Perspective Of Modeling Using Multirepresentationsmentioning

confidence: 99%

“…The NRC's Framework for K‐12 Science Education (NRC, 2012, pp. 56–57) reports that “scientists use models … to represent their current understanding of a system under study, to aid in the development of questions and explanations, and to communicate ideas to others.” Hestenes (1992, p. 732) argues that “the great game of science is modeling the real world.” Engaging students in developing and using models is considered a critical practice to improve students' scientific competence (Ke & Schwarz, 2021; Zhai, 2022; zu Belzen et al, 2019).…”

Section: An Inclusive Perspective Of Modeling Using Multirepresentationsmentioning

confidence: 99%

“…Scientists use a variety of representations—including models—to explain or predict phenomena. A scientific model includes both abstraction and representation of the critical features and mechanisms of phenomena (Zhai, 2022). It represents a system that explains or predict phenomena (Shemwell & Capps, 2019), and can take a variety of forms; these can be categorized based on features, such as having a representational approach (e.g., drawings, graphs, diagrams), an epistemic purpose (e.g., explanatory or predictive), or a computational approach (e.g., system models or agent‐based models; Harrison & Treagust, 2000).…”

Section: An Inclusive Perspective Of Modeling Using Multirepresentationsmentioning

confidence: 99%

“…Multirepresentations in assessment practices could elicit students' knowledge‐in‐use and provide enriched, inclusive means for students to present their understanding. Scientists frequently select multirepresentations as a communication “language” or a combination of “languages” to investigate phenomena (Lehrer & Schauble, 2015; Zhai, 2022). Such activities have inspired educators to involve students in constructing multirepresentations, to aid in assessing students with different learning capabilities.…”

Section: An Inclusive Perspective Of Modeling Using Multirepresentationsmentioning

confidence: 99%

See 3 more Smart Citations

Applying machine learning to automatically assess scientific models

Zhai

Krajcik

2022

J Res Sci Teach

Self Cite

View full text Add to dashboard Cite

Involving students in scientific modeling practice is one of the most effective approaches to achieving the next generation science education learning goals. Given the complexity and multirepresentational features of scientific models, scoring student‐developed models is time‐ and cost‐intensive, remaining one of the most challenging assessment practices for science education. More importantly, teachers who rely on timely feedback to plan and adjust instruction are reluctant to use modeling tasks because they could not provide timely feedback to learners. This study utilized machine learning (ML), the most advanced artificial intelligence (AI), to develop an approach to automatically score student‐drawn models and their written descriptions of those models. We developed six modeling assessment tasks for middle school students that integrate disciplinary core ideas and crosscutting concepts with the modeling practice. For each task, we asked students to draw a model and write a description of that model, which gave students with diverse backgrounds an opportunity to represent their understanding in multiple ways. We then collected student responses to the six tasks and had human experts score a subset of those responses. We used the human‐scored student responses to develop ML algorithmic models (AMs) and to train the computer. Validation using new data suggests that the machine‐assigned scores achieved robust agreements with human consent scores. Qualitative analysis of student‐drawn models further revealed five characteristics that might impact machine scoring accuracy: Alternative expression, confusing label, inconsistent size, inconsistent position, and redundant information. We argue that these five characteristics should be considered when developing machine‐scorable modeling tasks.

show abstract

Section: An Inclusive Perspective Of Modeling Using Multirepresentationsmentioning

confidence: 99%

Section: An Inclusive Perspective Of Modeling Using Multirepresentationsmentioning

confidence: 99%

Section: An Inclusive Perspective Of Modeling Using Multirepresentationsmentioning

confidence: 99%

Section: An Inclusive Perspective Of Modeling Using Multirepresentationsmentioning

confidence: 99%

See 2 more Smart Citations

Applying machine learning to automatically assess scientific models

Zhai

Krajcik

2022

J Res Sci Teach

Self Cite

View full text Add to dashboard Cite

show abstract

“…We believe that engaging students in drawing models could be a more equitable means for science learning as it minimizes linguistic practices. It is well acknowledged that modeling is less about linguistics and more about the representation of thinking (Zhai, 2022a). Science educators are working hard to ensure that students with lower language proficiencies are included in meaningful science learning, and pictorial modeling might be one solution.…”

Section: Ai Can Help Diversify Representational Modalities In Science...mentioning

confidence: 99%

AI and formative assessment: The train has left the station

Zhai

Nehm

2023

J Res Sci Teach

Self Cite

View full text Add to dashboard Cite

In response to Li, Reigh, He, and Miller's commentary, Can we and should we use artificial intelligence for formative assessment in science, we argue that artificial intelligence (AI) is already being widely employed in formative assessment across various educational contexts. While agreeing with Li et al.'s call for further studies on equity issues related to AI, we emphasize the need for science educators to adapt to the AI revolution that has outpaced the research community. We challenge the somewhat restrictive view of formative assessment presented by Li et al., highlighting the significant contributions of AI in providing formative feedback to students, assisting teachers in assessment practices, and aiding in instructional decisions. We contend that AI‐generated scores should not be equated with the entirety of formative assessment practice; no single assessment tool can capture all aspects of student thinking and backgrounds. We address concerns raised by Li et al. regarding AI bias and emphasize the importance of empirical testing and evidence‐based arguments in referring to bias. We assert that AI‐based formative assessment does not necessarily lead to inequity and can, in fact, contribute to more equitable educational experiences. Furthermore, we discuss how AI can facilitate the diversification of representational modalities in assessment practices and highlight the potential benefits of AI in saving teachers’ time and providing them with valuable assessment information. We call for a shift in perspective, from viewing AI as a problem to be solved to recognizing its potential as a collaborative tool in education. We emphasize the need for future research to focus on the effective integration of AI in classrooms, teacher education, and the development of AI systems that can adapt to diverse teaching and learning contexts. We conclude by underlining the importance of addressing AI bias, understanding its implications, and developing guidelines for best practices in AI‐based formative assessment.

show abstract

Developing an instrument to examine students' analogical modeling competence: An example of electricity

Lin,

Chao

2023

Science Education

View full text Add to dashboard Cite

Science education reforms advocate modeling as a core practice in which analogy is a significant form and analogical modeling is a creative process for scientific explanation and discovery. This study adopts the self‐generated analogical modeling approach involving electricity, which considers all the modeling subprocesses and includes two subtopics (circuits and energy) with incompatible ontological presuppositions as a conceptual space to develop an analogical modeling competence (AMC) instrument. Two constructs, phenomenon‐related abductive premises (Phenomenon‐AbPr) and analogical modeling processes (AMP), that describe the increasing sophistication of students' AMC (i.e., Level 0 experience to Level 4 high‐order relation) are considered. The procedures utilize a construct‐based approach of developing an assessment to validate the underlying constructs and subtopics to develop the mapping between the target and the analogical models, design items, and determine outcome spaces. Data were collected from 294 Grade 6 students to examine their AMC. The item response theory technique based on the Rasch partial credit model was used to validate the two‐phase model. Student results revealed that the infit and outfit mean square values were between 0.75 and 1.33, indicating acceptable item fit. Results revealed that students scored highest on the open series circuit and lowest on electric energy in the Phenomenon‐AbPr phase. Most students demonstrated relational mapping in the analogical model validation (Level 3) stage, which performed best in the AMP phase. However, they struggled to transfer factors during the analogical model application and deployment (Level 0). Implications for science assessment and teaching are discussed.

show abstract

Assessing high‐school students' modeling performance on Newtonian mechanics

Cited by 9 publications

References 100 publications

Applying machine learning to automatically assess scientific models

Applying machine learning to automatically assess scientific models

AI and formative assessment: The train has left the station

Developing an instrument to examine students' analogical modeling competence: An example of electricity

Contact Info

Product

Resources

About