Generalizable Automatic Short Answer Scoring via Prototypical Neural Network

A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students’ Formative Assessment Responses in Science

“…Their ability to compose paper outlines, expository essays, and screenplays, has made the use of ChatGPT ubiquitous across academia, business, and news media. In March 2023, Ope-nAI released GPT-4 3 (OpenAI 2023), which is largely considered the current state-of-the-art for LLMs (OpenAI 2023;Zhao et al 2023). For this reason, we chose to use GPT-4 as the LLM to develop and evaluate our approach.…”

“…Advances in natural language processing (NLP) have produced improved automated assessment scoring approaches to support teaching and learning (e.g., Adair et al 2023;Wilson et al 2021). Proposed methodologies include data augmentation , next sentence prediction (Wu et al 2023), prototypical neural networks (Zeng et al 2023), cross-prompt fine-tuning (Funayama et al 2023), human-in-the-loop scoring via sam-pling responses (Singla et al 2022), and reinforcement learning (Liu et al 2022). While these methods have enjoyed varying degrees of success, a majority of these applications have targeted more structured mathematics and computer science tasks (i.e., tasks that can be solved formulaically), but their grading is different from scoring free-form shortanswer responses by middle school students in science domains.…”