Improving Automated Evaluation of Student Text Responses Using GPT-3.5 for Text Data Augmentation

Abstract

In education, intelligent learning environments allow students to choose how to tackle open-ended tasks while monitoring performance and behavior, allowing for the creation of adaptive support to help students overcome challenges. Timely feedback is critical to aid students' progression toward learning and improved problem-solving. Feedback on text-based student responses can be delayed when teachers are overloaded with work. Automated evaluation can provide quick student feedback while easing the manual evaluation burden for teachers in areas with a high teacher-to-student ratio. Current methods of evaluating student essay responses to questions have included transformer-based natural language processing models with varying degrees of success. One main challenge in training these models is the scarcity of data for studentgenerated data. Larger volumes of training data are needed to create models that perform at a sufficient level of accuracy. Some studies have vast data, but large quantities are difficult to obtain when educational studies involve student-generated text. To overcome this data scarcity issue, text augmentation techniques have been employed to balance and expand the data set so that models can be trained with higher accuracy, leading to more reliable evaluation and categorization of student answers to aid teachers in the student's learning progression. This paper examines the text-generating AI model, GPT-3.5, to determine if prompt-based text-generation methods are viable for generating additional text to supplement small sets of student responses for machine learning model training. We augmented student responses across two domains using GPT-3.5 completions and used that data to train a multilingual BERT model. Our results show that text generation can improve model performance on small data sets over simple self-augmentation.

Extending a Pretrained Language Model (BERT) using an Ontological Perspective to Classify Students’ Scientific Expertise Level from Written Responses

“…An illustrative example of this application is evident in the work of , wherein LR models were employed to identify the causal structure present in students' scienti c explanations. Transformer-based Natural Language Processing (NLP) models, as exempli ed by prominent instances such as BERT and GPT, have become the de facto industry standard for a diverse range of NLP downstream tasks (Cochran, Cohn, Rouet, and Hastings, 2023;Wulff et al, 2023). Prior research (e.g., Cochran et al, 2022) has consistently highlighted the effectiveness of BERT-based transformers in evaluating students' responses to STEM-related questions.…”

How artificial intelligence affects education?

“…Whether and how the development of artificial intelligence will affect the process of education (AIED -Artificial Intelligence in Education) and skill acquisition is still under consideration. While one gets the impression that the revolution has already begun, as academic articles cite numerous examples of AI-based learning platforms (Cukrova et al, 2023) or student text assessment tools (Cochran et al, 2023) the public does not seem to control either the direction or pace of change.…”