Learning Analytics for Online Discussions: Embedded and Extracted Approaches

Abstract

This paper describes an application of learning analytics that builds on an existing research program investigating how students contribute and attend to the messages of others in asynchronous online discussions. We first overview the E-Listening research program and then explain how this work was translated into analytics that students and instructors could use to reflect on their discussion participation. Two kinds of analytics were designed: some embedded in the learning environment to provide students with real-time information on their activity in-progress; and some extracted from the learning environment and presented to students in a separate digital space for reflection. In addition, we describe the design of an intervention though which use of the analytics can be introduced as an integral course activity. Findings from an initial implementation of the application indicated that the learning analytics intervention supported changes in students' discussion participation. Five issues for future work on learning analytics in online discussions are presented. One, unintentional versus purposeful change; two, differing changes prompted by the same analytic; three, importance of theoretical buy-in and calculation transparency for perceived analytic value; four, affective components of students' reactions; and five, support for students in the process of enacting analytics-driven changes.

Analysis and Reflections on the Third Learning Analytics and Knowledge Conference (LAK 2013)

“…Wise, Zhao, and Hausknecht (2013) draw our attention to the importance of "invisible behaviour," such as student message reading, as well as message writing in online forums. They also discuss a framework for applying learning analytics that includes their integration with the goals of learning, student agency in interpreting diverse measures, providing time and space for learner reflection on analytics, and supporting discussion between students and instructors as partners.…”

Learning within Digital Media: Investigating the Relationships Between Student Citation Networks, Assignment Structures, and Learning Outcomes

“…Learning analytics investigates student data (everything from actual assignments and grades to trace data such as number of logins or time spent on an activity), seeking to identify patterns that can improve the educational experience or learning outcomes [15,29].…”

The quantified self for learning: critical questions for education

“…These include: how to determine what can and should be measured in order to support the learning process (Wise, Zhao, and Hausknecht 2013); how to develop measures that are relatively light touch, because if it is too much effort people will not provide enough data to the system (Rivera-Pelayo et al 2012); how best to develop systems which enable individuals to engage with their data and/or with the support of others to ensure maximum learning benefit (Harfield 2014); and how to address legal and ethical issues when combining individual's data from multiple data sets.…”

“…Relatedly, the datafication of learning could also unintentionally shift the focus of reflecting on learning away from learning itself and instead on to the measures (Wise, Zhao, and Hausknecht 2013). Such measures could distract from learning as individuals start to focus on the countable aspects of the learning process, which can act as a comfortable diversion.…”

“…Hours spent revising, numbers of words written per day, multiple choice questions answered in half an hour, can all become the most important metric, rather than the quality of the writing, the mathematical thinking or the cognitive process. To try to prevent a focus on the data, discussions of metrics with educated others (Harfield 2014), or using the data as part of some form of wider learning programme (Wise, Zhao, and Hausknecht 2013), could both be useful strategies. However, data have a captivating and powerful quality, as any assessor who wishes their students looked more at their feedback than the final mark will attest.…”