Background Engineers are increasingly learning from online courses, in school and the workplace, using online social and informational networks for instruction and to access expertise. Small effective changes in online learning can have large cumulative impact through‐out an organization. Purpose (Hypothesis) This study is a part of a research effort between industry and learning scientists to develop an evidence‐based methodology for online learning. We investigate how learning outcomes are impacted by online courses designed to support feedback and formative assessment during learning. Design/Method We compared two versions of an online course derived from identical resources. The new course (treatment) used integrated formative assessments to provide feedback to students during learning; focused video resources to anchor the formative assessments; and an instructional design process based on essential concepts to organize the overall design. The course was implemented with a freely available Learning Management System (LMS). Results Treatment course participants learned more overall than did control course participants, with more positive attitudes towards the course content and their future learning. Learning had less dependence on initial knowledge. The collaboration capabilities of the LMS supported course development, but had limited spontaneous use by students during the experiment. Conclusions The paper demonstrates that formative feedback and design methods which stress essential concepts can make significant differences in learning outcomes in online courses, a straightforward format for implementation, and ways of measuring effectiveness. Areas for further work are integrating feedback with authentic tasks and mining the data produced by student's online activities.

No abstract