Automated Personalized Feedback Improves Learning Gains in An Intelligent Tutoring System

Kochmar, Ekaterina; Vu, Dung Do; Belfer, Robert; Gupta, Varun; Serban, Iulian Vlad; Pineau, Joëlle

doi:10.1007/978-3-030-52240-7_26

Cited by 35 publications

(21 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our approach contrasts considerably with existing feedback models, such as Rus, Niraula, and Banjade (2015) and Kochmar et al (2020). While these systems can capably guide students towards fixing incorrect ideas, they cannot directly comment on which concepts the students have misunderstood.…”

Section: Introductionmentioning

confidence: 93%

“…Many ITSs provide feedback guiding students towards the correct solution in an exercise (Ramachandran et al 2018;Rus, Niraula, and Banjade 2015;Nye, Graesser, and Hu 2014;Ventura et al 2018;Al-Nakhal and Abu-Naser 2017;Al-Dahdooh and Abu-Naser 2017;Tamura et al 2015;Guo et al 2016;Shah, Shah, and Kurup 2017;Serban et al 2020;Kochmar et al 2020). However, to our knowledge, none have previously employed a neural discourse-based mechanism for personalized feedback.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Deep Discourse Analysis for Generating Personalized Feedback in Intelligent Tutor Systems

Grenander¹,

Belfer²,

Kochmar³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We explore creating automated, personalized feedback in an intelligent tutoring system (ITS). Our goal is to pinpoint correct and incorrect concepts in student answers in order to achieve better student learning gains. Although automatic methods for providing personalized feedback exist, they do not explicitly inform students about which concepts in their answers are correct or incorrect. Our approach involves decomposing students answers using neural discourse segmentation and classification techniques. This decomposition yields a relational graph over all discourse units covered by the reference solutions and student answers. We use this inferred relational graph structure and a neural classifier to match student answers with reference solutions and generate personalized feedback. Although the process is completely automated and data-driven, the personalized feedback generated is highly contextual, domain-aware and effectively targets each student's misconceptions and knowledge gaps. We test our method in a dialogue-based ITS and demonstrate that our approach results in high-quality feedback and significantly improved student learning gains.

show abstract

Section: Introductionmentioning

confidence: 93%

Section: Related Workmentioning

confidence: 99%

Deep Discourse Analysis for Generating Personalized Feedback in Intelligent Tutor Systems

Grenander¹,

Belfer²,

Kochmar³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Those are collected in a knowledge base in which TF-IDF and lexicon-based features techniques are applied to provide valuable insights to improve the overall teaching quality and methodology. Kochmar et al (2020) presented automatically generated personalized feedback. The proposal uses the Korbit learning platform from which automated and personalized feedback is generated considering students' individual needs; thus, the feedback is without expert mediation or predefined rules.…”

Section: Literature Reviewmentioning

confidence: 99%

A feedback model applied in a ubiquitous microlearning environment using SECA rules

et al. 2022

View full text Add to dashboard Cite

Understanding learners’ behavior is the key to the success of any learning process. The more we know about them, the more likely we can personalize learning experiences and provide successful feedback. This paper presents a feedback model implemented in a ubiquitous microlearning environment based on contextual and behavioral information and evaluation results. The model uses SECA rules where the Scenario (S) represents the ubiquitous context variables reflecting the learner behavior during the learning process. The Event (E) identifies the probability that a learner fails or passes its evaluation. Condition (C) evaluates the results of the events. Moreover, Action (A) provides feedback to the learner. The proposal is developed through a controlled experiment whereby a microlearning environment can collect data from a ubiquitous context. The feedback model applies an analytics process to find the best context and behavior variables through different classification models. Those models predict whether a learner could fail, determine evaluation results’ causes, and provide feedback. The Random Forest was the model with the best performance. Thus, 94% accuracy, a 97% Recall, a 93% Precision, an F1 score of 95%, and a Jaccard of 91%. Hence, each scenario is defined from a branch of every tree obtained from the Random Forest model personalizing feedback actions applying clustering techniques. Finally, we presented an exemplified set of feedback rules, providing automatic recommendations and improving learner experiences. Thus, the experiment allows analyzing the learner behavior in a ubiquitous microlearning context from a feedback perspective.

show abstract

“…Personalized natural language feedback can be used for such reinforcement, which has been shown to lead to substantial learning gains. 21 Furthermore, as the competent learner is capable of filtering extraneous information and stimuli, increased fidelity and immersion become appropriate at this stage of learning:It is herein recommended that the more immersive end of the XR continuum, where AR, MR, and VR are combined, would be most suitable to support advancing trainees from the advanced beginner to competent stage.…”

Section: Strategy For Incorporating Xr Technology Into Trainingmentioning

confidence: 99%

Performance gains from adaptive eXtended Reality training fueled by artificial intelligence

Stanney

Archer

Skinner

et al. 2021

Journal of Defense Modeling & Simulation

View full text Add to dashboard Cite

While virtual, augmented, and mixed reality technologies are being used for military medical training and beyond, these component technologies are oftentimes utilized in isolation. eXtended Reality (XR) combines these immersive form factors to support a continuum of virtual training capabilities to include full immersion, augmented overlays that provide multimodal cues to personalize instruction, and physical models to support embodiment and practice of psychomotor skills. When combined, XR technologies provide a multi-faceted training paradigm in which the whole is greater than the sum of the constituent capabilities in isolation. When XR applications are adaptive, and thus vary operational stressors, complexity, learner assistance, and fidelity as a function of trainee proficiency, substantial gains in training efficacy are expected. This paper describes a continuum of XR technologies and how they can be coupled with numerous adaptation strategies and supportive artificial intelligence (AI) techniques to realize personalized, competency-based training solutions that accelerate time to proficiency. Application of this training continuum is demonstrated through a Tactical Combat Casualty Care training use case. Such AI-enabled XR training solutions have the potential to support the military in meeting their growing training demands across military domains and applications, and to provide the right training at the right time.

show abstract

Automated Personalized Feedback Improves Learning Gains in An Intelligent Tutoring System

Cited by 35 publications

References 16 publications

Deep Discourse Analysis for Generating Personalized Feedback in Intelligent Tutor Systems

Deep Discourse Analysis for Generating Personalized Feedback in Intelligent Tutor Systems

A feedback model applied in a ubiquitous microlearning environment using SECA rules

Performance gains from adaptive eXtended Reality training fueled by artificial intelligence

Contact Info

Product

Resources

About