Explanations from ChatGPT: Targeted Task Improvements But No Generalized Gains in Metacognitive Abilities for Novice Language Learners

Abstract

This study investigated the potential benefits of ChatGPT’s explainability for enhancing self-regulated learning in novice language learners. Specifically, explanations from ChatGPT were explored as a tool to improve metacognitive awareness and task performance. 50 intermediate Iranian English learners engaged with either an explainable version of ChatGPT that justified its feedback or a non-explainable version without elucidations. Quantitative results showed no significant differences between groups on standardized measures of metacognitive knowledge or writing performance, indicating explainable ChatGPT did not improve these skills. However, the explainable ChatGPT group demonstrated significantly higher accuracy on target tasks covered in the system’s explanations. These mixed findings suggest explainable ChatGPT may strengthen understanding of specific content explained but do not confer generalized gains in metacognitive abilities or standardized outcomes. While explainable ChatGPT shows promise for supporting self-regulated learning, determining optimal applications to boost broad learner competencies remains an open research challenge. Further inquiry with larger diverse samples is warranted to fully ascertain the pedagogical potential of explainable ChatGPT in education.