When the Tutor Becomes the Student: Design and Evaluation of Efficient Scenario-based Lessons for Tutors

Thomas, Danielle R.; Yang, Xinyu; Gupta, Shivang; Adeniran, Adetunji; McLaughlin, Elizabeth A.; Koedinger, Ken

doi:10.1145/3576050.3576089

Cited by 11 publications

(11 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study, most students were paired based on pairing of advanced students with less advanced students based on the teacher's judgment of dashboard recommendations [35,70], which leaves room to investigate the benefits of masterybased pairings in future work. Upfront training of the Tutors has been shown to improve tutoring practice, which is expected to improve the learning of tutees [59]. Training could include a tutor effectiveness score via live analytics and simplified grading of Solver steps to free up cognitive resources (and feedback based on classifications of chat messages).…”

Section: Discussionmentioning

confidence: 99%

Combining Dialog Acts and Skill Modeling: What Chat Interactions Enhance Learning Rates During AI-Supported Peer Tutoring?

Borchers,

Yang,

Lin

et al. 2024

Preprint

View full text Add to dashboard Cite

Peer tutoring can improve learning by prompting learners to reflect. To assess whether peer interactions are conducive to learning and provide peer tutoring support accordingly, what tutorial dialog types relate to student learning most? Advancements in collaborative learning analytics allow for merging machine learning-based dialog act classification with cognitive modeling of fine-grained learning processes during problem-solving to illuminate this question. We estimate how much peer-tutored students improve in a collaborative tutoring system for linear equation-solving in K-12 mathematics in relationship to the peer dialog types they engage in. This work establishes a reliable BERT classifier with an accuracy of close to 80\% to classify chat messages during peer tutoring into minimal, facilitative, and constructive, serving as instructional factors. Based on data from 394 students, peer tutor dialog was rare. Only 8\% of tutee problem-solving steps were followed by peer tutor chat messages. Still, facilitative tutor dialog was associated with an increased tutee learning rate. Meanwhile, tutor dialog classified as constructive was associated with lower learning rates. Content analysis suggested that such dialog often reinforced incorrect solutions, gave away answers, or was unrelated to the taught content. Hence, considering problem-solving solution contexts could improve the assessment of peer tutoring dialog. Peer tutors engaging in little dialog could be attributed to the high cognitive demand of learning to tutor while still learning the content they tutor on. Providing peer tutors with instructional support to engage in constructive dialog may improve the tutee's learning.

show abstract

Section: Discussionmentioning

confidence: 99%

Combining Dialog Acts and Skill Modeling: What Chat Interactions Enhance Learning Rates During AI-Supported Peer Tutoring?

Borchers,

Yang,

Lin

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Our tutor training encompasses lessons grounded in the SMART framework [4], which identifies key domains research finds critical for tutoring success: Social-emotional Learning, Mastery of Content, Advocacy, Relationship-building, and Technology [28].…”

Section: Tutor Training and Lesson Developmentmentioning

confidence: 99%

“…The lessons task tutors with predicting and explaining the most desired approach when responding to common tutoring situations in both a pretest and posttest, involving analogous scenarios [5][6]. Considerable evidence suggests that tutors benefit from scenario-based lesson instruction, focusing on specific tutor skills, with tutors performing significantly better at the posttest compared to the pretest when presented with similar tutoring scenarios [29]. These lessons use a learn-by-doing approach known to be effective for acquiring new skills [16].…”

Section: Tutor Training and Lesson Developmentmentioning

confidence: 99%

“…Effective tutoring addresses students' challenges to motivation and engagement alongside mastering content and pedagogy [6,27,28]. To expand tutoring programs successfully, comprehensive, and ongoing training for adult tutors is needed, especially among volunteers, college students, and peers, who may lack pedagogical expertise and need support in relationship building [17].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Using Generative AI to Provide Feedback to Adult Tutors in Training and Assess Real-life Performance

Thomas,

Gatz,

Gupta

et al. 2024

Preprint

View full text Add to dashboard Cite

Tutoring remains among the most impactful academic interventions known to improve student achievement. Despite its success, there are few available opportunities for training and professional development for adult tutors. Furthermore, the task of assessing tutor performance during actual tutoring sessions is challenging and time-consuming for human evaluators—enter artificial intelligence (AI). In this work, we harness generative AI, specifi-cally large language models (LLMs) like GPT-4, to introduce an innovative approach for delivering tutors real-time, explanatory feedback while tutors engage in online scenario-based lessons. Hosted within the Personalized Learning Squared (PLUS) tutoring platform, these lessons offer tutors op-portunities to practice responding to common tutoring scenarios. For exam-ple, in the Giving Effective Praise lesson, tutors practice responding to students by providing praise and then receiving immediate and templated feedback generated by LLMs. Beyond demonstrating tutor training contain-ing AI-generated feedback, we report past work using LLMs to assess tutors providing praise, revealing moderate performance, and explaining several prompting methods. While using generative AI shows promise as a low-cost and efficient method for giving adult tutors feedback on their perfor-mance in training and actual tutoring, it is not without limitations. Practical and ethical considerations are discussed. Human tutoring remains a robust and irreplaceable influence on student learning, with generative AI serving as a valuable tool to complement tutors and enhance the impact of tutoring on student learning.

show abstract

“…elements of effective praise and encouraging tutor reactions to student errors (Thomas et al, 2023), ensuring that SPOCK provides immediate, earned, truthful, specific, and genuine feedback focused on the learning process.…”

Section: Introductionmentioning

confidence: 99%

CLASS: A Design Framework for Building Intelligent Tutoring Systems Based on Learning Science principles

Sonkar,

Liu,

Mallick

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

We present a design framework called Conversational Learning with Analytical Step-by-Step Strategies (CLASS) for building advanced Intelligent Tutoring Systems (ITS) powered by high-performance Large Language Models (LLMs). The CLASS framework empowers ITS with two key capabilities. First, through a carefully curated scaffolding dataset, CLASS equips ITS with essential problemsolving strategies, enabling it to provide tutorlike, step-by-step guidance to students. Second, by using a dynamic conversational dataset, CLASS assists ITS in facilitating natural language interactions, fostering engaging studenttutor conversations. The CLASS framework also provides valuable insights into ITS's internal decision-making process which allows seamless integration of user feedback, thus enabling continuous refinement and improvement. We also present a proof-of-concept ITS, referred to as SPOCK, which is trained using the CLASS framework with a focus on introductory college-level biology content. A carefully constructed protocol was developed for SPOCK's preliminary evaluation, examining aspects such as the factual accuracy and relevance of its responses. Experts in the field of biology offered favorable remarks, particularly highlighting SPOCK's capability to break down questions into manageable subproblems and provide encouraging responses to students.

show abstract

When the Tutor Becomes the Student: Design and Evaluation of Efficient Scenario-based Lessons for Tutors

Cited by 11 publications

References 12 publications

Combining Dialog Acts and Skill Modeling: What Chat Interactions Enhance Learning Rates During AI-Supported Peer Tutoring?

Combining Dialog Acts and Skill Modeling: What Chat Interactions Enhance Learning Rates During AI-Supported Peer Tutoring?

Using Generative AI to Provide Feedback to Adult Tutors in Training and Assess Real-life Performance

CLASS: A Design Framework for Building Intelligent Tutoring Systems Based on Learning Science principles

Contact Info

Product

Resources

About