What Does Methodology Mean for Learning Analytics?

Abstract

The diversity of substantive expertise and analytical techniques spanned by learning analytics can be a challenge for understanding and interrogating methodological choices. But active pursuit of methodological clarity is essential to productive multivocality. This was the focus of the LAK '17 Methodology in Learning Analytics workshop from which the idea for this special section arose. Discussions were framed using a building metaphor of developing structurally sound bridges between disciplinary methods. Strong arguments are buttressed by supporting evidence and warrants but also engage with potential alternative explanations. Papers in this special section further the discussion and encompass a range of relevant approaches including sensitivity analysis, model evaluation, model fit, causal inference, and visualization as a methodology. All advance the case of structural integrity in this bridge discipline and highlight the importance of adequate descriptions of methodological considerations to aid our common understanding.

Capturing the participation and social dimensions of computer-supported collaborative learning through social network analysis: which method and measures matter?

“…Few studies have been devoted to the examination of the role of different network configurations (Fincham et al 2018; in the field of education. Recently, LA researchers have started to address this gap (Bergner et al 2018). Fincham et al (2018), for example, examined the influence of different tie extraction methods on a network structure and statistical metrics.…”

“…The contextual variability is a widely recognized aspect in the field of LA in general and in SNA studies in particular (Bergner et al 2018;Joksimovic et al 2016). Interactions between students, teachers and learning tools in a course frequently vary by context and/or instructional conditions (Gašević et al 2016;Rogers et al 2016).…”

“…For a theory to advance our understanding of the complex nature of learning and teaching processes, there is an evident need for valid measurements and testable models that can deliver reproducible results (Loken and Gelman 2017;Smaldino and McElreath 2016). As stressed by Bergner et al (2018), "It can safely be assumed that without foregrounding methodological choices in learning analytics we run the risk of generating more doubt" (p. 3).…”

From theory to action

“…If LA are to be used as a methodological tool, greater focus must be placed on their development. Bergner et al [2] note that methodologies are positioned at the intersection of the "what" and the "how". Consequently, using LA as a methodology helps to interrogate what is being studied and how it is being studied.…”

Conflicting Values: An Exploration of the Tensions between Learning Analytics and Academic Librarianship

“…Clow (2013) notes that learning analytics takes an "eclectic approach" and that the field lacks a coherent epistemological basis. In addition, writing in a special issue on methodology and learning analytics, Bergner, Gray, and Lang (2018) recognize that while learning analytics draws on a wide variety of expertise, methods, and different disciplines, methodological clarity is necessary for the field to mature, define boundaries, and make and support claims. The use of learning analytics in higher education and particularly academic libraries provides the opportunity for critical reflection on the ways in which learning analytics conflicts with professional library values; core competencies, particularly research competency; ethics; and the broader purpose of education.…”