Evaluating and improving adaptive educational systems with learning curves

Martin, Brent; Mitrović, Antonija; Koedinger, Kenneth R.; Mathan, Santosh

doi:10.1007/s11257-010-9084-2

Cited by 68 publications

(51 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For the evaluation of large topics as knowledge assessment units, we have applied learning curve analysis (Martin et al 2011). Multiple experiments provide strong evidence that on the cognitive level, learning process generally follows the power law Newell,Rosenbloom 1981).…”

Section: Evaluation Of Topic-based User Modeling: Topic As An Assessmmentioning

confidence: 99%

“…The assumption behind the evaluation approach based on learning curve analysis is that learning curves that better approximate power law correspond to more cognitively valid units of knowledge. This approach has been pioneered in several projects (Koedinger,Mathan 2004) and formally suggested as an evaluation approach in (Martin et al 2005). Since that, it has been used in a number of publications to assess the quality of domain modeling.…”

Section: Evaluation Of Topic-based User Modeling: Topic As An Assessmmentioning

confidence: 99%

“…Since that, it has been used in a number of publications to assess the quality of domain modeling. A good summary of this approach could be found in (Martin et al 2011).…”

Section: Evaluation Of Topic-based User Modeling: Topic As An Assessmmentioning

confidence: 99%

See 2 more Smart Citations

Evaluation of topic-based adaptation and student modeling in QuizGuide

Sosnovsky

Brusilovsky

2015

User Model User-Adap Inter

View full text Add to dashboard Cite

Abstract:This paper presents an in-depth analysis of a nonconventional topic-based personalization approach for adaptive educational systems (AES) that we have explored for a number of years in the context of university programming courses. With this approach both student modeling and adaptation are based on coarse-grained knowledge units that we called topics. Our motivation for the topic-based personalization was to enhance AES transparency for both teachers and students by utilizing typical topic-based course structures as the foundation for designing all aspects of an AES from the domain model to the end-user interface. We illustrate the details of the topic-based personalization technology, with the help of the Webbased educational service QuizGuide -the first system to implement it. QuizGuide applies the topicbased personalization to guide students to the right learning material in the context of an undergraduate C programming course. While having a number of architectural and practical advantages, the suggested coarse-grained personalization approach deviates from the common practices toward knowledge modeling in AES. Therefore, we believed, that several aspects of QuizGuide required a detailed evaluation -from modeling accuracy to the effectiveness of adaptation. How can one evaluate the soundness of a user modeling (UM) approach, rather than a specific personalized system? The paper discusses how this new student modeling approach can be evaluated, and presents our attempts to evaluate it from multiple different prospects. The evaluation of QuizGuide across several consecutive semesters demonstrates that, although topics do not always support precise UM, they can provide a basis for successful personalization in AESs. Keywords:Adaptive Hypermedia, Student Modeling, Adaptive Educational System, Adaptive Navigation Support, Adaptive Link Annotation, Topic-based Adaptation, Topic-based User Modeling, Adaptive System Evaluation, Layered EvaluationThe first page should include the following declaration: "This paper or a similar version is not currently under review by a journal or conference, nor will it be submitted to such within the next three months. This paper is free of plagiarism or self-plagiarism as defined in Springer's Policy on Publishing Integrity. Abstract:This paper presents an in-depth analysis of a nonconventional topic-based personalization approach for adaptive educational systems (AES) that we have explored for a number of years in the context of university programming courses. With this approach both student modeling and adaptation are based on coarse-grained knowledge units that we called topics. Our motivation for the topic-based personalization was to enhance AES transparency for both teachers and students by utilizing typical topic-based course structures as the foundation for designing all aspects of an AES from the domain model to the end-user interface. We illustrate the details of the topic-based personalization technology, with the help of the Web-based educational service Quiz...

show abstract

Section: Evaluation Of Topic-based User Modeling: Topic As An Assessmmentioning

confidence: 99%

Section: Evaluation Of Topic-based User Modeling: Topic As An Assessmmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of topic-based adaptation and student modeling in QuizGuide

Sosnovsky

Brusilovsky

2015

User Model User-Adap Inter

View full text Add to dashboard Cite

show abstract

“…In this context, Martin et al [2] proposed to use socalled 'learning curves' to analyze and improve systems and models for personalized adaptive learning. A learning curve is a plot of the performance on a task versus the number of opportunities to practice either on an individual or a group basis (see Figure 1).…”

Section: Special Focus Paper Application Of Learning Curves For Didacmentioning

confidence: 99%

“…The tools iScale [10] and its variant UX curve [11] are used to capture users' evolving experiences with an inter- Figure 1. Two sample learning curves [2], one for an individual student (right-hand side) and the other one for a group of learners (lefthand side) active product/service over a longer-term duration (cf. episodic usability evaluation).…”

Section: Special Focus Paper Application Of Learning Curves For Didacmentioning

confidence: 99%

Application of Learning Curves for Didactic Model Evaluation: Case Studies

Mödritscher

Andergassen

Law

et al. 2013

Int. J. Emerg. Technol. Learn.

View full text Add to dashboard Cite

Abstract-The success of (online) courses depends, among other factors, on the underlying didactical models which have always been evaluated with qualitative and quantitative research methods. Several new evaluation techniques have been developed and established in the last years. One of them is 'learning curves', which aim at measuring error rates of users when they interact with adaptive educational systems, thereby enabling the underlying models to be evaluated and improved. In this paper, we report how we have applied this new method to two case studies to show that learning curves are useful to evaluate didactical models and their implementation in educational platforms. Results show that the error rates follow a power law distribution with each additional attempt if the didactical model of an instructional unit is valid. Furthermore, the initial error rate, the slope of the curve and the goodness of fit of the curve are valid indicators for the difficulty level of a course and the quality of its didactical model. As a conclusion, the idea of applying learning curves for evaluating didactical model on the basis of usage data is considered to be valuable for supporting teachers and learning content providers in improving their online courses.

show abstract

Data mining and education

Koedinger

D’Mello

McLaughlin

et al. 2015

WIRES Cognitive Science

Self Cite

101

View full text Add to dashboard Cite

An emerging field of educational data mining (EDM) is building on and contributing to a wide variety of disciplines through analysis of data coming from various educational technologies. EDM researchers are addressing questions of cognition, metacognition, motivation, affect, language, social discourse, etc. using data from intelligent tutoring systems, massive open online courses, educational games and simulations, and discussion forums. The data include detailed action and timing logs of student interactions in user interfaces such as graded responses to questions or essays, steps in rich problem solving environments, games or simulations, discussion forum posts, or chat dialogs. They might also include external sensors such as eye tracking, facial expression, body movement, etc. We review how EDM has addressed the research questions that surround the psychology of learning with an emphasis on assessment, transfer of learning and model discovery, the role of affect, motivation and metacognition on learning, and analysis of language data and collaborative learning. For example, we discuss (1) how different statistical assessment methods were used in a data mining competition to improve prediction of student responses to intelligent tutor tasks, (2) how better cognitive models can be discovered from data and used to improve instruction, (3) how data-driven models of student affect can be used to focus discussion in a dialog-based tutoring system, and (4) how machine learning techniques applied to discussion data can be used to produce automated agents that support student learning as they collaborate in a chat room or a discussion board.

show abstract

Evaluating and improving adaptive educational systems with learning curves

Cited by 68 publications

References 16 publications

Evaluation of topic-based adaptation and student modeling in QuizGuide

Evaluation of topic-based adaptation and student modeling in QuizGuide

Application of Learning Curves for Didactic Model Evaluation: Case Studies

Data mining and education

Contact Info

Product

Resources

About