What Do You Mean by Collaboration Analytics? A Conceptual Model

Abstract

Using data to generate a deeper understanding of collaborative learning is not new, but automatically analyzing log data has enabled new means of identifying key indicators of effective collaboration and teamwork that can be used to predict outcomes and personalize feedback. Collaboration analytics is emerging as a new term to refer to computational methods for identifying salient aspects of collaboration from multiple group data sources for learners, educators, or other stakeholders to gain and act upon insights. Yet, it remains unclear how collaboration analytics go beyond previous work focused on modelling group interactions for the purpose of adapting instruction. This paper provides a conceptual model of collaboration analytics to help researchers and designers identify the opportunities enabled by such innovations to advance knowledge in, and provide enhanced support for, collaborative learning and teamwork. We argue that mapping from low-level data to higher-order constructs that are educationally meaningful, and that can be understood by educators and learners, is essential to assessing the validity of collaboration analytics. Through four cases, the paper illustrates the critical role of theory, task design, and human factors in the design of interfaces that inform actionable insights for improving collaboration and group learning.

Incorporating AI and learning analytics to build trustworthy peer assessment systems

“…Specifically, the paper—through the conceptualisation of the four phases of peer‐review process—demonstrated how a theoretical model can be used to structure the program of research, development, deployment, and evaluation by addressing a problem (ie, trust in a peer‐review system) that may emerge in practice. This is an important contribution that complements existing calls for integration of theory and design with computational methods of AI and data science in learning analytics (Gašević et al, 2017; Martinez‐Maldonado et al, 2021). Second, the studies reported in the paper provide fresh empirical insights that can inform the development of future AI‐driven learning analytic systems that seek to enhance trustworthiness of peer‐review.…”

How Can High-Frequency Sensors Capture Collaboration? A Review of the Empirical Links between Multimodal Metrics and Collaborative Constructs

“…To our knowledge, there have been no attempts to empirically map out the entire space between data, meric, outcome, and constructs, which is difficult to achieve not only due to the sheer range in metrics and outcomes used, but also due to the diverse ways that metrics and outcomes are connected [11,16]. Our empirical survey of theory use in MMCA research is an additional novel contribution that responds to the call for more theory integration in social analytics [9,17].…”

“…Our work differentiates itself from prior reviews by scope, looking at MMCA papers in both education and social computing. While there is work in both fields that shares the same goal of detecting and supporting collaboration [9], there is little communication between the two fields, and reviews either focus on multimodal learning analytics (MMLA; e.g., [4,10]), or social computing research on collaboration (e.g., [11,12]). Additionally, prior reviews in MMLA or social computing presented models of research (e.g., [13,14]) or surveyed the types of collaborative outcomes that research focused on [10,11], or organized the types of data sources used [9,15].…”

“…While there is work in both fields that shares the same goal of detecting and supporting collaboration [9], there is little communication between the two fields, and reviews either focus on multimodal learning analytics (MMLA; e.g., [4,10]), or social computing research on collaboration (e.g., [11,12]). Additionally, prior reviews in MMLA or social computing presented models of research (e.g., [13,14]) or surveyed the types of collaborative outcomes that research focused on [10,11], or organized the types of data sources used [9,15]. To our knowledge, there have been no attempts to empirically map out the entire space between data, meric, outcome, and constructs, which is difficult to achieve not only due to the sheer range in metrics and outcomes used, but also due to the diverse ways that metrics and outcomes are connected [11,16].…”

Educational dialogues and computer supported collaborative learning: critical analysis and research perspectives

“…-Caveat: recent research on learning analytics does in fact make links between "e-data" (e.g., from group artefacts, speech and groupware logs) and specific higher-order constructs such as symmetry of action and transactivity in collaboration (e.g., Martinez-Maldonado et al, 2021); but these constructs, as well as initial data analysis, need to be situated within and guided by more general theories of dialogue, cognition and collective activity.…”