Distributing epistemic functions and tasks—A framework for augmenting human analytic power with machine learning in science education research

Abstract

Machine learning (ML) has become commonplace in educational research and science education research, especially to support assessment efforts. Such applications of machine learning have shown their promise in replicating and scaling human‐driven codes of students' work. Despite this promise, we and other scholars argue that machine learning has not yet achieved its transformational potential. We argue that this is because our field is currently lacking frameworks for supporting creative, principled, and critical endeavors to use machine learning in science education research. To offer considerations for science education researchers' use of ML, we present a framework, Distributing Epistemic Functions and Tasks (DEFT), that highlights the functions and tasks that pertain to generating knowledge that can be carried out by either trained researchers or machine learning algorithms. Such considerations are critical decisions that should occur alongside those about, for instance, the type of data or algorithm used. We apply this framework to two cases, one that exemplifies the cutting‐edge use of machine learning in science education research and another that offers a wholly different means of using machine learning and human‐driven inquiry together. We conclude with strategies for researchers to adopt machine learning and call for the field to rethink how we prepare science education researchers in an era of great advances in computational power and access to machine learning methods.

When a machine detects student reasoning: a review of machine learning-based formative assessment of mechanistic reasoning

“…The identified advancements and shortcomings of our analysis are in alignment with other reviews and perspectives (Zhai et al, 2020a(Zhai et al, , 2020cKubsch et al, 2023). In their dimension construct, Zhai et al (2020a) found that 81% of the reviewed studies tapped complex constructs indicating the great potential of ML in assessing such constructs.…”

“…Along with other researchers (Zhai et al, 2020a(Zhai et al, , 2020cKubsch et al, 2023), we suggest exploring the transformative character of ML in science education beyond automating formative assessment. This includes moving away from perceiving humans as consumers of algorithmic output that takes certain decision-making processes away since such output rarely answers research questions related to the qualitative analysis of cognitive operations (Kubsch et al, 2023). In other words, ML algorithms do not replace human qualitative interpretation; thoughtful human interpretation is more crucial than ever.…”

“…Subsequently, additional data sources can be combined to extend the validity of conclusions (Kubsch et al, 2021). To date, research has mostly focused on assessment automation, which only changes the quantity of research efforts, but not its quality (Nelson, 2020;Kubsch et al, 2023). Hence, we get only more, but the same insights as in traditional assessments.…”

“…Focusing on automation leaves many ML methods unexplored, which is why the quality of evidence from data does not change. In the future, it is, for instance, necessary to integrate computational techniques with interpretative human coding to augmentatively raise the validity of both and gain new insights into cognitive processes (Sherin, 2013;Nelson, 2020;Nelson et al, 2021;Kubsch et al, 2023).…”

Lehre in der organischen Chemie individualisieren

“…Diese ML-Techniken erlauben der Hochschulfachdidaktik, was bislang nur schwer möglich war: Sie bekommt durch die automatisierte Auswertung offener Aufgabenformate datengetriebene Einblicke in die Kompetenzen von Studierenden. 8)…”