The benefits and caveats of using clickstream data to understand student self-regulatory behaviors: opening the black box of learning processes

Baker, Rachel; Xu, Di; Park, Jihyun; Yu, Renzhe; Li, Qiujie; Cung, Bianca; Fischer, Christian; Rodrı́guez, Fernando; Warschauer, Mark; Smyth, Padhraic

doi:10.1186/s41239-020-00187-1

Cited by 65 publications

(50 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are several reasons that might explain the non-significant findings. First of all, although clickstream data enable researchers to gather fine-grained information about students' interactions with the device, concerns have been raised that this data source only provides a partial and noisy record of a student's actions (Baker et al, 2020). Also, during the intervention, we found that some page loading errors occured to some students while they were reading the WKe-Book.…”

Section: Non-significant Findings For the Time And Frequency-related Log Variablesmentioning

confidence: 93%

Using Interactive E-Book User Log Variables to Track Reading Processes and Predict Digital Learning Outcomes

Yang

Zargar

Adams

et al. 2020

Assessment for Effective Intervention

View full text Add to dashboard Cite

Stealth assessment has been successfully embedded in educational games to measure students’ learning in an unobtrusive and supportive way. This study explored the possibility of applying stealth assessment in a digital reading platform and sought to identify potential in-system indicators of students’ digital learning outcomes. Utilizing the user log data from third- to fifth-grade students ( n = 573) who read an interactive Word Knowledge E-Book, we examined various user log variables and their associations with word knowledge and strategic reading outcomes. Descriptive analysis provided a depiction of the real-time reading processes and behaviors in which students engaged while digitally reading. Multiple regression analysis with classroom fixed effects demonstrated that user log variables relevant to answering questions and making decisions (i.e., percentage of embedded questions answered correctly; number of attempts to answer the questions; and making implausible decisions) were significantly associated with students’ word knowledge and strategic reading outcomes. Variables indicating reading time and frequency, however, were not significantly associated with these outcomes. This study highlights the potential of interactive e-books as another digital learning environment to establish stealth assessment, which may allow researchers and educators to track students’ reading processes and predict reading outcomes while supporting digital learning.

show abstract

Section: Non-significant Findings For the Time And Frequency-related Log Variablesmentioning

confidence: 93%

Using Interactive E-Book User Log Variables to Track Reading Processes and Predict Digital Learning Outcomes

Yang

Zargar

Adams

et al. 2020

Assessment for Effective Intervention

View full text Add to dashboard Cite

show abstract

“…• Navigation time was computed as the cumulative time, in minutes, between web browser navigation events in the LMS, excluding any measured durations between navigation events greater than or equal to 25 minutes (which were likely to be periods of inactivity), and excluding durations between web browser sessions (Baker et al, 2020).…”

Section: Learning Management System (Lms) Recordsmentioning

confidence: 99%

A Pandemic of Busywork: Increased Online Coursework Following the Transition to Remote Instruction is Associated with Reduced Academic Achievement

Motz

Quick

Wernert

et al. 2021

OLJ

View full text Add to dashboard Cite

Under normal circumstances, when students invest more effort in their schoolwork, they generally show evidence of improved academic achievement. But when universities abruptly transitioned to remote instruction in Spring 2020, instructors assigned rapidly-prepared online learning activities, disrupting the normal relationship between effort and outcomes. In this study, we examine this relationship using data observed from a large-scale survey of undergraduate students, from logs of student activity in the online learning management system, and from students’ estimated cumulative performance in their courses (n = 4,636). We find that there was a general increase in the number of assignments that students were expected to complete following the transition to remote instruction, and that students who spent more time and reported more effort carrying out this coursework generally had lower course performance and reported feeling less successful. We infer that instructors, under pressure to rapidly put their course materials online, modified their courses to include online busywork that did not constitute meaningful learning activities, which had a detrimental effect on student outcomes at scale. These findings are discussed in contrast with other situations when increased engagement does not necessarily lead to improved learning outcomes, and in comparison with the broader relationship between effort and academic achievement.

show abstract

“…One promising method for understanding SRL in online learning is to observe students' behaviors within a course's Learning Management System (LMS) [5,26,37]. Some LMS services allow researchers to obtain student clickstream data, which are timestamped logs that record every single click event in the course, such as the specific page or resource a student visited.…”

Section: Introductionmentioning

confidence: 99%

“…Some LMS services allow researchers to obtain student clickstream data, which are timestamped logs that record every single click event in the course, such as the specific page or resource a student visited. A large online course of about 300 students can produce roughly 380,000 individual data points [5]. Because of the size and richness of the data, researchers have been able to take advantage of data mining techniques to develop sophisticated models of college student learning [16,23].…”

Section: Introductionmentioning

confidence: 99%

Using Clickstream Data Mining Techniques to Understand and Support First-Generation College Students in an Online Chemistry Course

Rodrı́guez

Lee

Rutherford

et al. 2021

LAK21: 11th International Learning Analytics and Knowledge Conference

Self Cite

View full text Add to dashboard Cite

Although online courses can provide students with a high-quality and flexible learning experience, one of the caveats is that they require high levels of self-regulation. This added hurdle may have negative consequences for first-generation college students. In order to better understand and support students' self-regulated learning, we examined a fully online Chemistry course with high enrollment (N = 312) and a high percentage of first-generation college students (65.70%). Using students' lecture video clickstream data, we created two indicators of self-regulated learning: lecture video completion and time management. Performing a k-means clustering on these indicators uncovered four distinct self-regulated learning patterns:(1) Early Planning, (2) Planning, (3) Procrastination, and (4) Low Engagement. Early Planning behaviors were especially important for course success-they consistently predicted higher final course grades, even after controlling for important demographic variables. Interestingly, first-generation college students classified as Early Planners achieved at similar levels as their non-first-generation peers, but first-generation students in the Low Engagement group had the lowest average grades among students. Overall, our results show that self-regulation may be an important skill for determining first-generation students' STEM achievement, and targeting these skills may serve as a useful way to support their specific learning needs.

show abstract

The benefits and caveats of using clickstream data to understand student self-regulatory behaviors: opening the black box of learning processes

Cited by 65 publications

References 64 publications

Using Interactive E-Book User Log Variables to Track Reading Processes and Predict Digital Learning Outcomes

Using Interactive E-Book User Log Variables to Track Reading Processes and Predict Digital Learning Outcomes

A Pandemic of Busywork: Increased Online Coursework Following the Transition to Remote Instruction is Associated with Reduced Academic Achievement

Using Clickstream Data Mining Techniques to Understand and Support First-Generation College Students in an Online Chemistry Course

Contact Info

Product

Resources

About