Comparing Human-Human to Human-Computer Tutorial Dialogue

Int J Artif Intell Educ

Steinhauser

Farrow

et al. 2014

Within STEM domains, physics is considered to be one of the most difficult topics to master, in part because many of the underlying principles are counter-intuitive. Effective teaching methods rely on engaging the student in active experimentation and encouraging deep reasoning, often through the use of selfexplanation. Supporting such instructional approaches poses a challenge for developers of Intelligent Tutoring Systems. We describe a system that addresses this challenge by teaching conceptual knowledge about basic electronics and electricity through guided experimentation with a circuit simulator and reflective dialogue to encourage effective self-explanation. The Basic Electricity and Electronics Tutorial Learning Environment (BEETLE II) advances the state of the art in dynamic adaptive feedback generation and natural language processing (NLP) by extending symbolic NLP techniques to support unrestricted student natural language input in the context of a dynamically changing simulation environment in a moderately complex domain. This allows contextually-appropriate feedback to be generated "on the fly" without requiring curriculum designers to anticipate possible student answers and manually author multiple feedback messages. We present the results of a system evaluation. Our curriculum is highly effective, achieving effect sizes of 1.72 when comparing pre-to post-test learning gains from our system to those of a no-training control group. However, we are unable to demonstrate that dynamically generated feedback is superior to a non-NLP feedback condition. Evaluation of interpretation quality demonstrates its link with instructional effectiveness, and provides directions for future research and development.

Section: Creating the Gold Standardmentioning

confidence: 99%

Section: Future Workmentioning

confidence: 99%

BEETLE II: Deep Natural Language Understanding and Automatic Feedback Generation for Intelligent Tutoring in Basic Electricity and Electronics

Int J Artif Intell Educ

Steinhauser

Farrow

et al. 2014

“…An initial study comparing 41 students in full to human tutors was presented in [25]. It demonstrated that there were significant differences in the distribution of metacognitive and social statements between full and human-human tutoring, as well as differences in which variables were correlated in learning gain.…”

Section: Resultsmentioning

confidence: 99%

“…The coding scheme we used is presented in Table 1 (reproduced from [25]). All student utterances were classified as primarily content, metacognitive, social or nonsense.…”

Section: Codingmentioning

confidence: 99%

“…First, we found evidence that students who produce a higher percentage of content words learn more [15] (see also [14] for a similar result in another tutoring system). Second, we showed that different forms of non-content talk (social and metacognitive statements) are correlated with learning in human-human dialogue and in human-computer dialogue [16,25]. However, different factors are (positively or negatively) correlated with learning gain in human-computer versus humanhuman dialogue.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Content, Social, and Metacognitive Statements: An Empirical Study Comparing Human-Human and Human-Computer Tutorial Dialogue

Sustaining TEL: From Innovation to Learning and Practice

Steinhauser

Moore

et al. 2010

Self Cite

We present a study which compares human-human computermediated tutoring with two computer tutoring systems based on the same materials but differing in the type of feedback they provide. Our results show that there are significant differences in interaction style between human-human and human-computer tutoring, as well as between the two computer tutors, and that different dialogue characteristics predict learning gain in different conditions. We show that there are significant differences in the non-content statements that students make to human and computer tutors, but also to different types of computer tutors. These differences also affect which factors are correlated with learning gain and user satisfaction. We argue that ITS designers should pay particular attention to strategies for dealing with negative social and metacognitive statements, and also conduct further research on how interaction style affects human-computer tutoring.

Intelligent Tutoring with Natural Language Support in the Beetle II System

Sustaining TEL: From Innovation to Learning and Practice

Bental

Moore

et al. 2010

Self Cite

We present Beetle II, a tutorial dialogue system designed to accept unrestricted language input and support experimentation with different tutorial planning and dialogue strategies. Our first system evaluation used two different tutoring policies and demonstrated that Beetle II can be successfully used as a platform to study the impact of different approaches to tutoring. In the future, the system can also be used to experiment with a variety of parameters that may affect learning in intelligent tutoring systems.