Editorial: Advances in multimodal learning: pedagogies, technologies, and analytics

Abstract

The rapid development of digital technologies and data-driven techniques has led to advances in multimodal learning featured by multimodality in instructional stimuli, learning spaces, behavioral pattern, and data sources (Blikstein and Worsley, 2016;Di Mitri et al., 2018;Chango et al., 2021). The combination of multiple sensory stimuli (e.g., visual, audio, verbal, tactile, and olfactory) in instructional content is known to promote cognitive performance, sense of presence, and learning engagement (Moreno and Mayer, 1999;Marucci et al., 2021). The fusion of traditional and digital learning spaces enabled by online platforms and extended reality has created more personal, accessible, and risk-free learning conditions that may otherwise be unavailable (Garrison and Kanuka, 2004;Allcoat et al., 2021;Luo et al., 2021). Multimodality in learning also emphasizes the multiplicity of mode of behaviors such as communication, interaction, and regulation, leading to increased interests in their impacts on learning achievement and experience (Mangaroska et al., 2020;Cloude et al., 2022;Ninaus and Sailer, 2022). More importantly, Multimodal Learning Analytics (MMLA) allows traces of various cognitive, behavioral, and affective indicators to be extracted from multiple data sources (e.g., eye-tracking, wearable cameras, gesture recognition systems, infrared imaging, biosensors) to assist the measurement and understanding of complex learning processes (Blikstein and Worsley, 2016;Di Mitri et al., 2018;Chango et al., 2021).Consequently, to capture the unique benefits of multimodal learning, research attention needs to be paid to four key aspects of multimodality, including learning content, learning space, learning process, and learning analytics. This Research Topic is an attempt to address such a research need by illustrating recent research development of multimodal learning in the four aspects. It comprises a total of 15 articles contributed by 63 authors globally, from academic and research institutions in Mainland China,