Scalability, Sustainability, and Ethicality of Multimodal Learning Analytics

Abstract

Multimodal Learning Analytics (MMLA) innovations are commonly aimed at supporting learners in physical learning spaces through state-of-the-art sensing technologies and analysis techniques. Although a growing body of MMLA research has demonstrated the potential benefits of sensor-based technologies in education, whether their use can be scalable, sustainable, and ethical remains questionable. Such uncertainty can limit future research and the potential adoption of MMLA by educational stakeholders in authentic learning situations. To address this, we systematically reviewed the methodological, operational, and ethical challenges faced by current MMLA works that can affect the scalability and sustainability of future MMLA innovations. A total of 96 peer-reviewed articles published after 2010 were included. The findings were summarised into three recommendations, including i) improving reporting standards by including sufficient details about sensors, analysis techniques, and the full disclosure of evaluation metrics, ii) fostering interdisciplinary collaborations among experts in learning analytics, software, and hardware engineering to develop affordable sensors and upgrade MMLA innovations that used discontinued technologies, and iii) developing ethical guidelines to address the potential risks of bias, privacy, and equality concerns with using MMLA innovations. Through these future research directions, MMLA can remain relevant and eventually have actual impacts on educational practices.

From Sensor Data to Educational Insights

“…This can also make full consenting from the perspective of students, educators and other educational stakeholders more challenging, as it may be harder for them to fully understand the implications of capturing and analyzing each data modality [ 11 , 23 ]. Together, the added complexity in infrastructure and expertise required and informed consenting may threaten the long-term sustainability and scalability of sensor-based educational solutions, which has already been identified in a recent MMLA literature review [ 10 ]. In sum, much research is needed to develop sensor-based MMLA systems with integrity that balance the benefits of augmenting the learning situations with the potential ethical and practical challenges that this conveys.…”

“…Although numerous studies are emerging in the literature, it is still challenging to find practitioners using these technologies in day-to-day teaching. Therefore, it is hard to find significant findings that are robust enough to identify underlying dependencies across the explored variables or to replicate studies to see whether results are universal and which of them are context-dependent [ 10 ]. We need more resilient science in this context that can help overcome these current limitations.…”

“…Moreover, many new sensor-based tools, such as sensor-based games [ 7 ] or realistic laboratories [ 8 , 9 ], are being built to support the educational process. The challenge is embedding sensors and resulting data representations in authentic educational settings in pedagogically meaningful and ethical ways [ 10 ].…”

The role of indoor positioning analytics in assessment of simulation‐based learning

“…Multimodal learning analytics (MMLA) can potentially support the assessment and feedback provision process in simulation‐based learning by making salient aspects of students' learning behaviours visible for objective evaluation and for provoking reflection (Crescenzi‐Lanna, 2020; Sharma & Giannakos, 2020; Yan, Zhao, et al, 2022). Recent small‐scale studies in MMLA have used wearable positioning sensors to automatically capture students' positioning traces in simulation‐based learning (Echeverria et al, 2018; Fernandez‐Nieto, Martinez‐Maldonado, Echeverria, et al, 2021; Fernandez‐Nieto, Martinez‐Maldonado, Kitto, & Shum, 2021; Zhao et al, 2022).…”

“…This subfield of MMLA endeavours to generate evidence‐based insights about students' learning behaviours and cognitive and emotional states from fine‐grained physical and physiological traces (Blikstein, 2013). The data collection capabilities of various combinations of sensing technologies and analysis methodologies in educational research have been well established in prior MMLA studies (Crescenzi‐Lanna, 2020; Yan, Martinez‐Maldonado, et al, 2022; Yan, Zhao, et al, 2022). In addition, a recent systematic literature review also illustrated the potential benefits of MMLA technologies in supporting reflective teaching and learning practices (Sharma & Giannakos, 2020).…”