Few-Shot Question Generation for Personalized Feedback in Intelligent Tutoring Systems

Abstract

Existing work on generating hints in Intelligent Tutoring Systems (ITS) focuses mostly on manual and non-personalized feedback. In this work, we explore automatically generated questions as personalized feedback in an ITS. Our personalized feedback can pinpoint correct and incorrect or missing phrases in student answers as well as guide them towards correct answer by asking a question in natural language. Our approach combines cause–effect analysis to break down student answers using text similarity-based NLP Transformer models to identify correct and incorrect or missing parts. We train a few-shot Neural Question Generation and Question Re-ranking models to show questions addressing components missing in the student’s answers which steers students towards the correct answer. Our model vastly outperforms both simple and strong baselines in terms of student learning gains by 45% and 35% respectively when tested in a real dialogue-based ITS. Finally, we show that our personalized corrective feedback system has the potential to improve Generative Question Answering systems.

New Era of Artificial Intelligence in Education: Towards a Sustainable Multifaceted Revolution

“…Hleg et al [17] 2019 Presents ethics guidelines for trustworthy AI Terzopoulos et al [18] 2019 Explores the capabilities of voice assistants in the classroom Webber et al [19] 2019 Discusses the potential application of AI to improve teamwork Marcinkowski et al [20] 2020 Investigates algorithmic vs. human decision-making in HE admissions Ahmed et al [21] 2021 Investigates an LMS in entrepreneurship Borenstein et al [22] 2021 Examines the ethical implications of Artificial Intelligence technologies González-Calatayud et al [23] 2021 Analyzes the use of AI for student assessment Kamalov et al [24] 2021 Explores Machine Learning for exam-cheating detection Miao et al [25] 2021 Discusses AI in Education policy Timan et al [26] 2021 Discusses data protection in the age of AI UNESCO [27] 2021 Provides recommendations on the ethics of AI JISC [28] 2022 Discusses and reflects on AI in Tertiary Education Kulshreshtha et al [29] 2022 Explores automatically generated questions as personalized feedback in an ITS Long et al [30] 2022 Explores Collaborative Knowledge Tracing to predict students' correctness in answering questions Mishkin et al [31] 2022 Presents risks and limitations for DALL-E 2 Nguyen et al [32] 2022 Discusses ethical principles for AI in Education Oxford Insights Government AI Readiness Index [33] 2022 Compares how 160 governments are prepared to use AI in public services Qadir et al [34] 2022 Discusses advantages and drawbacks on Generative AI in education St-Hilaire et al [35] 2022 Presents the results of a comparative study on learning outcomes for two popular online learning platforms Swiecki et al [36] 2022 Discusses Generative AI and assessment practices Wahle et al [37] 2022 Explores detection of machine-paraphrased plagiarism AlAfnan et al [38] 2023 Explores advancements in Artificial Intelligence and its applications Bouschery et al [39] 2023 Focuses on product innovation management research and strategies Chan et al [40] 2023 Presents a preprint discussing specific topics related to teachers, AI and Higher Education Chen et al [41] 2023 Investigates the use of chatbots in classrooms Chetouani et al [42] 2023 Examines human-centered AI, human-c...…”

Opinerium: Subjective Question Generation Using Large Language Models

“…An early study proposed a method based on BERT transformer [38] which has gained attention. Eventually, with advances in the large language models, a personalized feedback question generation method [39] was introduced using T5 [40] and BART [41] transformer models. Similarly, GPT-2 was utilized for the question generation task [42].…”

Generative Agents to support students learning progress

“…Improvement in automatic scoring of student responses through pre-trained language models was investigated by . Kulshreshtha et al (Kulshreshtha et al, 2022) utilized the transformer architecture for the process of generating individualized feedback during tutoring. They take the answer from the students and a manual gold standard drafted answer to generate individual feedback for the student.…”

“…While multiple-choice questions are relatively easy to grade automatically, open-ended answers present a more complex problem for automated correction. Beyond just identifying the right or wrong responses, the system should also be capable of providing constructive feedback and guiding the students towards improved responses, and showing ways how questions could be answered in a more familiar way (Kulshreshtha et al, 2022).…”

“…Here, the system would not only indicate the errors but also guide the student to the correct understanding. This could involve the generation of partial answers or hints, suggesting relevant resources, or pinpointing the parts of the original text that the student should revisit, like it is proposed also in Kulshreshtha et al (Kulshreshtha et al, 2022).…”