Machine Learning and Hebrew NLP for Automated Assessment of Open-Ended Questions in Biology

Ariely, Moriah; Nazaretsky, Tanya; Alexandron, Giora

doi:10.1007/s40593-021-00283-x

Cited by 26 publications

(14 citation statements)

References 74 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…During the PDP, the teachers were presented with AI‐Grader—an NLP and AI‐powered assessment technology that analyzes constructed responses (CR) to open‐ended questions in Biology and generates an automated score based on an analytic grading rubric (Figure 1). The analytic grading rubric represents the required chain of reasoning, including all the main content and causal relations that should appear in a correct response (for a full description of the items, grading rubrics and the NLP‐based algorithms, see Ariely et al (2022).…”

Section: Methodsmentioning

confidence: 99%

“…The PDP developed in this study focused on a specific application of Natural Language Processing (NLP) and AI for automated grading of constructed responses in biology (Ariely et al, 2022), and had two main goals: (i) to promote teachers' understanding of causal explanations in biology and how to teach and assess written explanations, using analytic grading rubrics; and (ii) to promote teachers' understanding of AI‐EdTech, including affordances and limitations of automated assessment in comparison to human assessment. While this study centres on the second goal, these two goals are well‐connected.…”

Section: Introduction and Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Teachers' trust in AI‐powered educational technology and a professional development program to improve it

Nazaretsky

Ariely

Cukurova

et al. 2022

Brit J Educational Tech

Self Cite

151

View full text Add to dashboard Cite

Evidence from various domains underlines the critical role that human factors, and especially trust, play in adopting technology by practitioners. In the case of Artificial Intelligence (AI) powered tools, the issue is even more complex due to practitioners' AI‐specific misconceptions, myths and fears (e.g., mass unemployment and privacy violations). In recent years, AI has been incorporated increasingly into K‐12 education. However, little research has been conducted on the trust and attitudes of K‐12 teachers towards the use and adoption of AI‐powered Educational Technology (AI‐EdTech). This paper sheds light on teachers' trust in AI‐EdTech and presents effective professional development strategies to increase teachers' trust and willingness to apply AI‐EdTech in their classrooms. Our experiments with K‐12 science teachers were conducted around their interactions with a specific AI‐powered assessment tool (termed AI‐Grader) using both synthetic and real data. The results indicate that presenting teachers with some explanations of (i) how AI makes decisions, particularly compared to the human experts, and (ii) how AI can complement and give additional strengths to teachers, rather than replacing them, can reduce teachers' concerns and improve their trust in AI‐EdTech. The contribution of this research is threefold. First, it emphasizes the importance of increasing teachers' theoretical and practical knowledge about AI in educational settings to gain their trust in AI‐EdTech in K‐12 education. Second, it presents a teacher professional development program (PDP), as well as the discourse analysis of teachers who completed it. Third, based on the results observed, it presents clear suggestions for future PDPs aiming to improve teachers' trust in AI‐EdTech. What is already known about this topic Human factors, and especially trust, play a critical role in practitioners' adoption of technology. In recent years, AI has been incorporated increasingly into K‐12 education. Little research has been conducted on the trust and attitudes of K‐12 teachers towards the use and adoption of AI‐powered Educational Technology. What this paper adds This research emphasizes the importance of increasing teachers' theoretical and practical knowledge about AI in educational settings to gain their trust in AI‐EdTech in K‐12 education. It presents a teacher professional development program (PDP) to increase teachers' trust in AI‐EdTech, as well as the discourse analysis of teachers who completed it. It presents clear suggestions for future PDPs aiming at improving teachers' trust in AI‐EdTech. Implications for practice and/or policy Pre‐ and in‐service teacher education programs that aim to increase teachers' trust in AI‐EdTech should include a section providing teachers with a basic understanding of AI. PDPs aimed to increase teachers' trust in AI‐EdTech should focus on concrete pedagogical tasks and specific AI‐powered tools that are considered by teachers as helpful and worth the effort to learn. AI‐EdTech should not restr...

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introduction and Literature Reviewmentioning

confidence: 99%

Teachers' trust in AI‐powered educational technology and a professional development program to improve it

Nazaretsky

Ariely

Cukurova

et al. 2022

Brit J Educational Tech

Self Cite

151

View full text Add to dashboard Cite

show abstract

“…The instrument was constructed in English and Hebrew as part of our previous study [3]. One of the authors manually translated it into Turkish (Table 1).…”

Section: The Instrumentmentioning

confidence: 99%

“…However, for languages like Turkish, Hebrew, and Arabic, a combination of being morphologically rich (where each input token may consist of several functional units, e.g., multiple suffixes and prefixes added to the original word root), and relatively low resource in the educational domain, makes applications of NLP in such languages particularly challenging [24]. To our knowledge, little research exists in this area [1,3,5,4,7]. [3] proposed a method for automated formative assessment of scientific explanations in Hebrew based on analytic rubrics.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Towards Automated Assessment of Scientific Explanations in Turkish using Language Transfer

Nazaretsky¹,

Yolcu²,

Ariely³

et al. 2023

Preprint

View full text Add to dashboard Cite

The paper presents a preliminary study on employing Natural Language Processing (NLP) techniques for automated formative assessment of scientific explanations in Turkish, a morphologically rich language with limited educational resources. The proposed method employs zero and few-shot language transfer techniques for creating Turkish NLP models, obviating the need for extensive collection and annotation of Turkish datasets. The study utilizes multilingual BERT-based pre-trained transformer models. It evaluates the effectiveness of different fine-tuning approaches using an existing annotated dataset in Hebrew. The results indicate that, despite being trained using non-perfectly automated translations from Hebrew responses, the best-performing models demonstrated adequate performance when evaluated on authentic Turkish responses. Thus, this research may provide a useful method for building automated scientific explanations assessment models that are transferred between languages.

show abstract

Causal‐mechanical explanations in biology: Applying automated assessment for personalized learning in the science classroom

Ariely,

Nazaretsky,

Alexandron

2024

J Res Sci Teach

View full text Add to dashboard Cite

One of the core practices of science is constructing scientific explanations. However, numerous studies have shown that constructing scientific explanations poses significant challenges to students. Proper assessment of scientific explanations is costly and time‐consuming, and teachers often do not have a clear definition of the educational goals for formulating scientific explanations. Consequently, teachers struggle to support their students in this process. It is hoped that recent advances in machine learning (ML) and its application to educational technologies can assist teachers and learners in analyzing student responses and providing automated formative feedback according to well‐defined pedagogical criteria. In this study, we present a method to automate the entire assessment‐feedback process. First, we developed a causal‐mechanical (CM)‐based grading rubric and applied it to student responses to two open‐ended items. Second, we used unsupervised ML tools to identify patterns in student responses. Those patterns enable the definition of “meta‐categories” of explanation types and the design of personalized feedback adapted to each category. Third, we designed an in‐class intervention with personalized formative feedback that matches the response patterns. We used natural language processing and ML algorithms to assess students' explanations and provide feedback. Findings from a controlled experiment demonstrated that a CM‐based grading scheme can be used to identify meaningful patterns and inform the design of formative feedback that promotes student ability to construct explanations in biology. We discuss possible implications for automated assessment and personalized teaching and learning of scientific writing in K‐12 science education.

show abstract

Machine Learning and Hebrew NLP for Automated Assessment of Open-Ended Questions in Biology

Cited by 26 publications

References 74 publications

Teachers' trust in AI‐powered educational technology and a professional development program to improve it

Teachers' trust in AI‐powered educational technology and a professional development program to improve it

Towards Automated Assessment of Scientific Explanations in Turkish using Language Transfer

Causal‐mechanical explanations in biology: Applying automated assessment for personalized learning in the science classroom

Contact Info

Product

Resources

About