Integrating Model-Driven and Data-Driven Techniques for Analyzing Learning Behaviors in Open-Ended Learning Environments

Abstract

Research in computer-based learning environments has long recognized the vital role of adaptivity in promoting effective, individualized learning among students. Adaptive scaffolding capabilities are particularly important in open-ended learning environments, which provide students with opportunities for solving authentic and complex problems, and the choice to adopt a variety of strategies and approaches to solving these problems. To help students overcome their difficulties and become effective learners and problem solvers, we have to develop methods that can track and interpret students' open-ended learning and problem-solving behaviors. The complexity of the problems and the open-ended nature of the solution processes pose considerable challenges to accurately interpret and evaluate student behaviors and performance as they work on the system. In this paper, we develop a framework that combines model-driven strategy detection with data-driven pattern discovery for analyzing students' learning activity data in open-ended environments. We present results from an in-depth case study of multiple activity patterns identified in data from the Betty's Brain learning environment. The results illustrate the benefits of combining model-and data-driven techniques to precisely characterize the learning behavior of students in an open-ended environment.!

Big Educational Data & Analytics: Survey, Architecture and Challenges

“…identify whether the instructions were inadequate or insufficient, or to identify visibility problems in the content posted) in order to review and organize the educational content. The authors in [96] presented a framework for analyzing student activity data in open-ended learning environments (OELE) that integrates model-driven behavior characterization and data-driven pattern discovery. The model-driven approach used linked task and strategy models to provide more precise interpretation of student activity sequences as learning and problem-solving strategies while the pattern mining approach enables the identification of new variations of strategies and of gaps in the coverage of the current strategy model.…”

Defining and Assessing Students’ Computational Thinking in a Learning by Modeling Environment

“…To formally define students' behaviors in the learning environment, we use a hierarchical model to characterize students' actions in CTSiM. Following the combined theory-and data-driven framework developed from Coherence Analysis (CA) (Segedy, Kinnebrew, & Biswas, 2015;Kinnebrew et al, 2017), we define three types of primary sub-tasks in CTSiM: Information Acquisition (IA), Solution Creation (SC), and Solution Assessment (SA). These primary sub-tasks can be further decomposed.…”

“…Finally, we show how the Coherence Analysis-derived (Kinnebrew et al, 2017) metrics can help characterize students' application of STEM + CT practices. Coherence analysis provides a framework for defining a number of metrics related to individual tasks students perform in the system (e.g., seeking information, building models, and checking models) (Segedy et al, 2015).…”

Computer-Assisted Assessment in Open-Ended Activities through the Analysis of Traces: A Proof of Concept in Statistics with R Commander

“…Efforts are currently being made to face the problem of time-consuming assessment in order to achieve automatic grading in specific open-ended scenarios. For example, De Marsico, Sciarrone, Sterbini, and Temperini (2017) use a Bayesian network to predict students' performance using teacher and peer-evaluation in open-ended questions, and Kinnebrew, Segedy and Biswas (2017) interpret students' open-ended learning and problem-solving behaviors in Betty's Brain environment by using Hidden Markov Models and Differential Sequence Mining. Both approaches are difficult to generalize and provide results that are hard to interpret by an instructor leading a class.…”