Can we and should we use artificial intelligence for formative assessment in science?

Li, Tingting; Reigh, Emily; He, Peng; Miller, Emily Adah

doi:10.1002/tea.21867

Cited by 22 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accordingly, teachers and students would receive immediate and meaningful feedback to adjust their teaching and learning strategies to achieve the targeted learning goals (He et al, in press, c). While recognizing the ethics and transparency issues in applying AI technologies in education (Li, Reigh, et al, 2023), we envision generative AI as a promising and unexplored area for enhancing standards-aligned and coherent learning systems to better enhance student knowledge-in-use development across time.…”

Section: Initial Version Final Versionmentioning

confidence: 99%

Transforming standards into classrooms for knowledge-in-use: an effective and coherent project-based learning system

He,

Krajcik,

Schneider

2023

Discip Interdscip Sci Educ Res

Self Cite

View full text Add to dashboard Cite

Global science education reform calls for developing student knowledge-in-use that applies the integrated knowledge of core ideas and scientific practices to make sense of phenomena or solve problems. Knowledge-in-use development requires a long-term, standards-aligned, coherent learning system, including curriculum and instruction, assessment, and professional learning. This paper addresses the challenge of transforming standards into classrooms for knowledge-in-use and presents an iterative design process for developing a coherent and standards-aligned learning system. Using a project-based learning approach, we present a theory-driven, empirically validated learning system aligned with the U.S. science standards, consisting of four consecutive curriculum and instruction materials, assessments, and professional learning to support students’ knowledge-in-use in high school chemistry. We also present the iterative development and testing process with empirical evidence to support the effectiveness of our learning system in a five-year NSF-funded research project. This paper discusses the theoretical perspectives of developing an NGSS-aligned, coherent, and effective learning system and recaps the development and testing process by unpacking all essential components in our learning system. We conclude that our theory-driven and empirically validated learning system would inform high school teachers and researchers across countries in transforming their local science standards into curriculum materials to support students’ knowledge-in-use development.

show abstract

Section: Initial Version Final Versionmentioning

confidence: 99%

Transforming standards into classrooms for knowledge-in-use: an effective and coherent project-based learning system

He,

Krajcik,

Schneider

2023

Discip Interdscip Sci Educ Res

Self Cite

View full text Add to dashboard Cite

show abstract

“…The potential of AI and visualization for assessment design has garnered growing research interest (Bodily et al., 2018; Gogri et al., 2013; Li et al., 2023). Nonetheless, several of these studies have been conceptual and devoid of empirical evidence from impact assessments.…”

Section: Introductionmentioning

confidence: 99%

Design and implementation of an AI‐enabled visual report tool as formative assessment to promote learning achievement and self‐regulated learning: An experimental study

Liao,

Zhang,

Wang

et al. 2024

Brit J Educational Tech

View full text Add to dashboard Cite

Formative assessment is essential for improving teaching and learning, and AI and visualization techniques provide great potential for its design and delivery. Using NLP, cognitive diagnostic and visualization techniques designed to analyse and present students' monthly exam data, we developed an AI‐enabled visual report tool comprising six modules and conducted an empirical study of its effectiveness in a high school biology classroom. A total of 125 students in a ninth‐grade biology course were assigned to a treatment group (n = 63) receiving AI‐enabled visual reports as the intervention and a control group (n = 62) receiving overall oral feedback from the teacher. We present the main statistical results of the within‐subjects design and the between‐subjects design respectively, to better capture the main findings. Repeated measures ANOVA revealed a significant interaction effect of intervention and time on learning achievement, and the paired‐sample Wilcoxon test indicated that the treatment group had experienced increasing learning anxiety (Cohen's d = 0.203, p = 0.046) and self‐efficacy (Cohen's d = 1.793, p = 0.000) over time. Moreover, we conducted a series of non‐parametric tests to compare the effects of AI‐enabled visual reports and teacher feedback, but found no significant differences except for an increased self‐efficacy (Cohen's d = 0.312, p = 0.046). Additionally, we had the students in the treatment group rate their favourable modules in the AI‐enabled visual report and provide evaluative feedback. The study results provide important insights into the design and implementation of effective formative assessment supported by artificial AI and visualization techniques. Practitioner notesWhat is already known about this topic Formative assessment is essential for improving teaching and learning. Traditional formative assessment tools lack accurate data‐oriented assessment and usability. AI and visualization techniques have great potential for formative assessment. What this paper adds This study designs and implements an AI‐enabled visual report tool that generates data‐driven, user‐friendly reports. The AI‐enabled visual report can not only enhance students' learning achievement and self‐regulated learning over time but also increase their test anxiety. The AI‐enabled visual report has a comparable effect with teacher feedback but leads to increased self‐efficacy. Implications for practice and/or policy We recommend using the AI‐enabled visual report in large‐size classes for its overall positive effects on both learning achievement and self‐regulated learning. We recommend using the AI‐enabled visual report over teacher feedback for its capacity to enhance students' self‐efficacy. We recommend prioritizing the modules of Performance Ranking, Personal Mastery and Knowledge Alert when designing the AI‐enabled visual report.

show abstract

“…Artificial Intelligence, Science Education, Formative Assessment, Bias, Machine Learning 1 | INTRODUCTION Li, Reigh, He, and Miller (2023) commented on our paper, Applying Machine Learning to Automatically Assess Scientific Models , and asked whether we can and should use artificial intelligence (AI) for formative assessment in science. We believe Li et al made a timely commentary given the advent and unfamiliarity of applying AI in science education.…”

Section: Introductionmentioning

confidence: 99%

“…In our view, this approach is akin to "throwing out the baby with the bathwater." The limitations and criticisms in Li et al (2023) directed at AI for formative assessment can be applied to almost all assessments.…”

Section: Introductionmentioning

confidence: 99%

AI and formative assessment: The train has left the station

Zhai

Nehm

2023

J Res Sci Teach

View full text Add to dashboard Cite

In response to Li, Reigh, He, and Miller's commentary, Can we and should we use artificial intelligence for formative assessment in science, we argue that artificial intelligence (AI) is already being widely employed in formative assessment across various educational contexts. While agreeing with Li et al.'s call for further studies on equity issues related to AI, we emphasize the need for science educators to adapt to the AI revolution that has outpaced the research community. We challenge the somewhat restrictive view of formative assessment presented by Li et al., highlighting the significant contributions of AI in providing formative feedback to students, assisting teachers in assessment practices, and aiding in instructional decisions. We contend that AI‐generated scores should not be equated with the entirety of formative assessment practice; no single assessment tool can capture all aspects of student thinking and backgrounds. We address concerns raised by Li et al. regarding AI bias and emphasize the importance of empirical testing and evidence‐based arguments in referring to bias. We assert that AI‐based formative assessment does not necessarily lead to inequity and can, in fact, contribute to more equitable educational experiences. Furthermore, we discuss how AI can facilitate the diversification of representational modalities in assessment practices and highlight the potential benefits of AI in saving teachers’ time and providing them with valuable assessment information. We call for a shift in perspective, from viewing AI as a problem to be solved to recognizing its potential as a collaborative tool in education. We emphasize the need for future research to focus on the effective integration of AI in classrooms, teacher education, and the development of AI systems that can adapt to diverse teaching and learning contexts. We conclude by underlining the importance of addressing AI bias, understanding its implications, and developing guidelines for best practices in AI‐based formative assessment.

show abstract

Can we and should we use artificial intelligence for formative assessment in science?

Cited by 22 publications

References 19 publications

Transforming standards into classrooms for knowledge-in-use: an effective and coherent project-based learning system

Transforming standards into classrooms for knowledge-in-use: an effective and coherent project-based learning system

Design and implementation of an AI‐enabled visual report tool as formative assessment to promote learning achievement and self‐regulated learning: An experimental study

AI and formative assessment: The train has left the station

Contact Info

Product

Resources

About