Moving Forward with Learning Analytics: Expert Views

Abstract

Learning analytics involve the measurement, collection, analysis, and reporting of data about learners and their contexts, in order to understand and optimize learning and the environments in which it occurs. Since emerging as a distinct field in 2011, learning analytics has grown rapidly, and early adopters around the world are developing and deploying these new tools. This paper reports on a study that investigated how the field is likely to develop by 2025 in order to recommend actions to those concerned with the implementation of learning analytics. The study used a Policy Delphi approach, presenting a range of future scenarios to international experts in the field and asking for responses related to the desirability and feasibility of these scenarios, as well as actions that would be required. Responses were received from 103 people from 21 countries. Responses were coded thematically, inter-rater reliability was checked using Cohen's kappa coefficient, and data were recoded if kappa was below 0.6. The seven major themes identified within the data were power, pedagogy, validity, regulation, complexity, ethics, and affect. The paper considers in detail each of these themes and its implications for the implementation of learning analytics. Notes for Practice• This paper analyzes responses from around the world and identifies seven factors that must be taken into account when implementing learning analytics: power, pedagogy, validity, regulation, complexity, ethics, and affect.• Responses revealed widespread unease about how analytics may develop. Bringing people together to engage with and understand the issues will be one way of addressing this problem.• It is also important for different communities to discuss and understand the value of their personal data and to know how they can be used, developed, and protected.• Evidence that learning analytics delivers benefits is not yet convincing to experts in the field.• Major investment of thoughtful effort is required in terms of research agendas and funding, policy and regulation, and developing and informing practice among all those who engage with learning analytics.

The efficacy of learning analytics interventions in higher education: A systematic review

“…However, most previous reviews have not focused specifically on the effectiveness of interventions based on LA. In fact, to our knowledge, only three reviews have been published with this particular focus (Ferguson & Clow, 2018;Ferguson et al, 2016;Viberg, Hatakka, Bälter, & Mavroudi, 2018). These papers provide good insight into the current state of the evidence (particularly Viberg et al, 2018 who review the Learning Analytics Community Exchange (LACE) hub of evidence) and make similar conclusions on this basis -that is, that there is limited empirical evidence to support the effectiveness of LA in higher education in terms of student outcomes.…”

Enabling Systematic Adoption of Learning Analytics through a Policy Framework

“…The development of protocols for the above mentioned activities were driven by the research questions listed above, and the methods used to analyse the collected data include cluster analysis, exploratory factor analysis, confirmatory factor analysis, and thematic content analysis. The development of the policy framework was inspired and guided by the Rapid Outcome Mapping Approach (ROMA) [1,2,4]. The ROMA model begins by defining an overarching policy objective, followed by six steps designed to provide policy makers with context-based information: (1) map political context, (2) identify key stakeholders, (3) identify desired behaviour changes, (4) develop engagement strategy, (5) analyse internal capacity to e↵ect change, and (6) establish monitoring and learning frameworks.…”

“…Our research project sets out to tackle with the identified problems by building a policy framework that is based on findings of various consultations with a diversity of stakeholders. The study aims to answer four questions: (1) what is the state of the art in terms of LA adoption in Europe, (2) what are the key challenges that impede institutional adoption of LA,…”

A Framework to Support Interdisciplinary Engagement with Learning Analytics

“…analytics. Learning analytics describes the process of not only collecting these data points but also analyzing and utilizing the data to inform decisions to improve and support student learning and success in educational contexts (Corrin et al, 2019;Ferguson et al, 2019). This type of distillation of the various data points into individualized student outcomes and impacts is one reason that learning analytics is an emerging field of study (de Freitas et al, 2015).…”

“…Learning analytics, first introduced circa 2011, has become an emerging field at institutions around the world (Ferguson, Clow, Griffiths, & Brasher, 2019). The emergence of learning analytics can be linked to the digitization of student recordsfrom activity within the learning management system (LMS) to efforts to identify patterns for retention and enrollment management -and has only served to make digital data an invaluable commodity for higher education institutions (Daniel, 2015;de Freitas et al, 2015;Prinsloo & Slade, 2018).…”