Is the doer effect a causal relationship?

Abstract

The "doer effect" is an association between the number of online interactive practice activities students' do and their learning outcomes that is not only statistically reliable but has much higher positive effects than other learning resources, such as watching videos or reading text. Such an association suggests a causal interpretation-more doing yields better learning-which requires randomized experimentation to most rigorously confirm. But such experiments are expensive, and any single experiment in a particular course context does not provide rigorous evidence that the causal link will generalize to other course content. We suggest that analytics of increasingly available online learning data sets can complement experimental efforts by facilitating more widespread evaluation of the generalizability of claims about what learning methods produce better student learning outcomes. We illustrate with analytics that narrow in on a causal interpretation of the doer effect by showing that doing within a course unit predicts learning of that unit content more than doing in units before or after. We also provide generalizability evidence across four different courses involving over 12,500 students that the learning effect of doing is about six times greater than that of reading.

Using online learner trace data to understand the cohesion of teams in higher education

“…These findings are not unexpected, as teams who seek frequent feedback from their industry supervisors are likely to achieve higher performance outcomes (Gabelica et al, 2012). Moreover, learners who practice engagement, as opposed to learners who read learning content, are likely to lead to better outcomes, also known as the ‘doers effect’ (Koedinger et al, 2016). These findings might also explain the poorer performance outcomes from teams in clusters 2 and 3 based on their lower engagement with feedback and practical assessments.…”

The Accurate Measurement of Students’ Learning in E-Learning Environments

“…At the end of the course, students took the final exam (post-test) consisting of 35 multiple-choice items. A more detailed description of this course can be found elsewhere [28,29]. Among the total of 27,720 students who registered the course, the data for our analysis consisted of 829 students (3%) who completed both the pre-and post-tests.…”

Designing High Structure Courses to Promote Student Engagement

“…Carnegie Mellon's Open Learning Initiative specializes in creating homework and quiz questions embedded in readings that give instant feedback to students. They estimate the learning benefit of their learn-by-doing activities to be six times the benefit of watching videos or reading alone (Koedinger et al 2016).…”

“…For example, activities that require students to practice simpler component skills will help them achieve more complex learning outcomes (Ambrose et al 2010). You may also simply emphasize that they will learn more by doing than watching or reading (Koedinger et al 2016). Simple reminders that you want all of your students to succeed can also be powerful, as students may not hear that enough.…”