Evaluation of an Automated Reading Tutor That Listens: Comparison to Human Tutoring and Classroom Instruction

Abstract

A year-long study of 131 second and third graders in 12 classrooms compared three daily 20-minute treatments. a) Fifty-eight students in six classrooms used the 1999-2000 version of Project LISTEN's Reading Tutor, a computer program that uses automated speech recognition to listen to a child read aloud, and gives spoken and graphical assistance. Students took daily turns using one shared Reading Tutor in their classroom while the rest of their class received regular instruction. b) Thirty-four students in the other six classrooms were pulled out daily for one-on-one tutoring by certified teachers. To control for materials, the human tutors used the same set of stories as the Reading Tutor. c) Thirty-nine students served as in-classroom controls, receiving regular instruction without tutoring. We compared students' preto post-test gains on the Word Identification, Word Attack, Word Comprehension, and Passage Comprehension subtests of the Woodcock Reading Mastery Test, and in oral reading fluency. Surprisingly, the human-tutored group significantly outgained the Reading Tutor group only in Word Attack (main effects p < .02, effect size .55). Third graders in both the computer-and human-tutored conditions outgained the control group significantly in Word Comprehension (p < .02, respective effect sizes .56 and .72) and suggestively in Passage Comprehension (p = .14, respective effect sizes .48 and .34). No differences between groups on gains in Word Identification or fluency were significant. These results are consistent with an earlier study in which students who used the 1998 version of the Reading Tutor outgained their matched classmates in Passage Comprehension (p = .11, effect size .60), but not in Word Attack, Word Identification, or fluency. To shed light on outcome differences between tutoring conditions and between individual human tutors, we compared process variables. Analysis of logs from all 6,080 human and computer tutoring sessions showed that human tutors included less rereading and more frequent writing than the Reading Tutor. Micro-analysis of 40 videotaped sessions showed that students who used the Reading Tutor spent considerable time waiting for it to respond, requested help more frequently, and picked easier stories when it was their turn. Human tutors corrected more errors, focused more on individual letters, and provided assistance more interactively, for example getting students to sound out words rather than sounding out words for students as the Reading Tutor did. EVALUATION OF AN AUTOMATED READING TUTOR THAT LISTENS / 63 EVALUATION OF AN AUTOMATED READING TUTOR THAT LISTENS / 65

On the Use of Automatic Speech Recognition to Facilitate Increased Literacy Rates in Jamaica

“…They therefore, did not provide an authentic language learning experience. However, recent advances have allowed for improvements that have led to these approaches being considered, in some spheres, to be cost-effective alternatives to employing experts to provide individual attention to learners [17].…”

“…ASR technology readily allows for the development of aids in which two-way interaction is achieved. An automatic reading tutor that listens to the user and provides useful, real-time feedback is more likely to replicate the experience of having a one-to-one session with a literacy specialist [17,20]. Among the potential gains is the fact that users would have multiple modes of stimulation, which holds the potential to enhance the effectiveness of the lesson.…”

“…McCandless for example, developed an ALT, where the user read a text shown on the computer screen and the tutor listened and provided feedback on what was said [16]. Another approach, Project LISTEN worked as a tutor by listening to, as well as reading to the user [17]. This system was one of the first to employ feedback that was able to effectively respond to readers when they encountered challenges or made mistakes.…”

Verification System for Quran Recitation Recordings

“…Many research efforts have been done for improving such systems especially in the field of second language teaching [1] [7][17] [15] [6]. Computer programs can be an important complement to language teachers for pronunciation training and accent improvements in foreign language classes [16]. No one can deny the use of computer in automated tutor has.…”

Toward unobtrusive measurement of reading comprehension using low-cost EEG

“…And though they have shown significant correlation with standardized comprehension tests [2], there remains room for improvement.…”