Coping with Poor Advice from Peers in Peer-Based Intelligent Tutoring: The Case of Avoiding Bad Annotations of Learning Objects

Abstract

Abstract. In this paper, we examine a challenge that arises in the application of peer-based tutoring: coping with inappropriate advice from peers. We examine an environment where students are presented with those learning objects predicted to improve their learning (on the basis of the success of previous, like-minded students) but where peers can additionally inject annotations. To avoid presenting annotations that would detract from student learning (e.g. those found confusing by other students) we integrate trust modeling, to detect over time the reputation of the annotation (as voted by previous students) and the reputability of the annotator. We empirically demonstrate, through simulation, that even when the environment is populated with a large number of poor annotations, our algorithm for directing the learning of the students is effective, confirming the value of our proposed approach for student modeling. In addition, the research introduces a valuable integration of trust modeling into educational applications.

From Small Seeds Grow Fruitful Trees: How the PHelpS Peer Help System Stimulated a Diverse and Innovative Research Agenda over 15 Years

“…In a seminal paper (VanLehn et al 1994) first proposed simulation as important for AIED, but the bulk of subsequent work in the field has been on simulated pedagogical agents. Simulation for testing AIED systems and exploring pedagogical questions is just now becoming important, with both high fidelity models such as SimStudent (Matsuda et al 2007) and low fidelity models as in (Champaign et al 2011). In our lab we have embarked on a lifelong learning project (Lelei and McCalla 2015) to build a simulated graduate school environment (that will run for 10 simulated years) in which to explore how various kinds of peer help can affect graduate students.…”

Demonstrating the value of credibility modeling for trust-based approaches to online message recommendation

“…This captures the fact that non-similar peers can still deliver useful information; perfect negative correlations are just as informative as positive ones. We formalize this combination as follows: (2) p EP Here, P is the set of all peers. This combination capitalizes on the fact that the beta distribution is well-defined for all real valued parameters a, (3 > O.…”

“…Champaign et al [2] develop a model for recommending commentary (annotations) on learning objects (texts or videos) to users in an online learning environment, which we term LOAR (Learning Object Annotation Recommendation). The model displays those annotations with the highest predicted learning benefits.…”

Validating trust models against realworld data sets

“…A sample of messages from this data set can be viewed in the Appendix. 5 To draw a distinction between "Common Ratings" and "Advisors", consider the case where a user A has rated only 1 message. It may be the case that all other users also rated that message, in which case the user has a large number of advisors (for that message).…”

“…In these experiments, we compare three approaches for classifying messages: LOAR [5], BLADE [3], and our POMDP model. We show the LOAR results as a baseline estimate of classification accuracy, as it models ratings from peers and rater similarity to users, both.…”