Improving Student Learning with Hybrid Human-AI Tutoring: A Three-Study Quasi-Experimental Investigation

Abstract

Artificial intelligence (AI) applications to support human tutoring have potential to significantly improve learning outcomes, but engagement issues persist, especially among students from low-income backgrounds. We introduce an AI-assisted tutoring model that combines human and AI tutoring and hypothesize this synergy will have positive impacts on learning processes. To investigate this hypothesis, we conduct a three-study quasi-experiment across three urban and low-income middle schools: 1) 125 students in a Pennsylvania school; 2) 385 students (50% Latinx) in a California school, and 3) 75 students (100% Black) in a Pennsylvania charter school, all implementing analogous tutoring models. We compare learning analytics of students engaged in human-AI tutoring compared to students using math software only. We find human-AI tutoring has positive effects, particularly in student's proficiency and usage, with evidence suggesting lower achieving students may benefit more compared to higher achieving students. We illustrate the use of quasi-experimental methods adapted to the particulars of different schools and data-availability contexts so as to achieve the rapid data-driven iteration needed to guide an inspired creation into effective innovation. Future work focuses on improving the tutor dashboard and optimizing tutor-student ratios, while maintaining annual costs per student of approximately $700 annually.CCS Concepts: • Human-centered computing → Human computer interaction (HCI); • Applied computing → Education.

Differential effects of GPT-based tools on comprehension of standardized passages

“…Together, these ndings support our two key pre-registered hypotheses on the effects of AI tools. Additionally, these ndings corroborate previous literature in that Intelligent Tutoring Systems (ITS) disproportionately help lower performers over higher performers 24,25 To rule out the possibility of regression to the mean (which is low due to our randomized design), we conducted complementary analyses using different low -high performer splits, splitting on SAT/ACT scores (taken 1-5 years prior to the current study) and overall performance on all tests (including AI conditions).…”