Capturing the participation and social dimensions of computer-supported collaborative learning through social network analysis: which method and measures matter?

Abstract

The increasing use of digital learning tools and platforms in formal and informal learning settings has provided broad access to large amounts of learner data, the analysis of which has been aimed at understanding students' learning processes, improving learning outcomes, providing learner support as well as teaching. Presently, such data has been largely accessed from discussion forums in online learning management systems and has been further analyzed through the application of social network analysis (SNA). Nevertheless, the results of these analyses have not always been reproducible. Since such learning analytics (LA) methods rely on measurement as a first step of the process, the robustness of selected techniques for measuring collaborative learning activities is critical for the transparency, reproducibility and generalizability of the results. This paper presents findings from a study focusing on the validation of critical centrality measures frequently used in the fields of LA and SNA research. We examined how different network configurations (i.e., multigraph, weighted, and simplified) influence the reproducibility and robustness of centrality measures as indicators of student learning in CSCL settings. In particular, this research aims to contribute to the provision of robust and valid methods for measuring and better understanding of the participation and social dimensions of collaborative learning. The study was conducted based on a dataset of 12 university courses. The results show that multigraph configuration produces the most consistent and robust centrality measures. The findings also show that degree centralities calculated with the multigraph methods are reliable indicators for students' participatory efforts as well as a consistent predictor of their performance. Similarly, Eigenvector centrality was the most consistent centrality that reliably represented social dimension, regardless of the network configuration. This study offers guidance on the appropriate network representation as well as sound recommendations about how to reliably select the appropriate metrics for each dimension.

Social practices in teacher knowledge creation and innovation adoption: a large-scale study in an online instructional design community for inquiry learning

“…For instance, the use of community analytics (Klamma 2013;Jeong et al 2017) could contribute to guide further analyses of the Go-Lab community focusing on the teachers themselves, how they move from individual to collaborative interactions, and how these interactions develop over time (Imroz 2019). From a network analysis point of view, integrating social network analysis (Jan et al 2019;Saqr et al 2020), social learning analytics (Shum and Ferguson 2012), and cluster analysis could help us understand the social practices and the role that experts play in them (Martínez et al 2006;Risser and Bottoms 2014). Finally, including additional data sources beyond the technical ecosystem (e.g., recording the discussions during teacher training sessions, or gathering the teachers' perceptions about their learning experience) would significantly enrich the initial quantitative estimations reported in our study, as already illustrated by Leoste et al (2019).…”

Using Three Social Network Analysis Approaches to Understand Computer-Supported Collaborative Learning

“…Fincham et al (2018) used different social tie extraction methods and examined the influences on the structural and statistical properties of the networks; the results confirmed that social tie definitions play an important role in shaping the results. Saqr et al (2020) examined how different network configurations influence the reproducibility and robustness of centrality measures as indicators of CSCL learning. Therefore, researchers should be aware that relational ties can be built in varied ways, and the choices of measurements can influence results.…”

“…Diverse types of “actors” are involved in, such as learners, technologies, and activities; relations are built between these “actors”, which have critical influences on CSCL processes. Therefore, an underlying philosophy of CSCL is “relations matter” (Dado & Bodemer, 2017; Saqr et al., 2020; Wu & Nian, 2021). To understand CSCL, it is necessary to analyze characteristics of CSCL entities, the relations between them, and the mediating effect of these relations on CSCL processes and outcomes.…”

“…Here, in this research, I particularly focus on the use of SNA under the exploratory analysis category to understand and describe CSCL from three main perspectives: relational ties , network modes , and integrated methods . First, SNA node-level and network-level metrics—grounded upon the measurements of relational ties—is a main SNA method used to understand attributes of entities and groups (e.g., Ouyang & Scharber, 2017; Saqr et al., 2020). Since relational ties can be built and measured through varied ways, it is necessary to explicitly elaborate definitions and measurements of relational ties (Ouyang & Scharber, 2017; Saqr et al., 2020; Wise & Cui, 2018).…”

“…First, SNA node-level and network-level metrics—grounded upon the measurements of relational ties—is a main SNA method used to understand attributes of entities and groups (e.g., Ouyang & Scharber, 2017; Saqr et al., 2020). Since relational ties can be built and measured through varied ways, it is necessary to explicitly elaborate definitions and measurements of relational ties (Ouyang & Scharber, 2017; Saqr et al., 2020; Wise & Cui, 2018). Second, two-mode and multi-mode network analyses are used to examine relations between entities, and mediating effects of these relations (e.g., Gao et al., 2017; Hecking et al., 2014; Malzahn et al., 2005).…”