Gaze-Driven Design Insights to Amplify Debugging Skills: A Learner-Centered Analysis Approach

Mangaroska, Katerina; Sharma, Kshitij; Giannakos, Michail N.; Trætteberg, Hallvard; Dillenbourg, Pierre

doi:10.18608/jla.2018.53.7

Cited by 12 publications

(4 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, each data stream carries a different set of noise sources, and hence, improving the signal-to-noise-ratio for each data stream and having it at similar levels might be a tedious task (Sharma, Papamitsiou, et al ., 2019 ). In addition, each data stream entails a separate type of features and measures (emotions from faces, D'Mello & Graesser, 2012 ; attention from eye-tracking, Mangaroska, Sharma, Giannakos, Traeteberg, & Dillenbourg, 2018 ; mental workload from EEG, Doppelmayr, Klimesch, Schwaiger, Auinger, & Winkler, 1998 ), and once the different measures and features are extracted from the collected, cleaned (noise-removal) and synchronized data streams, extracting LA-specific guidelines is another challenge (Bakharia et al ., 2016 ;Giannakos et al ., 2019 ). Given that most researchers on MMLA rely on custom-developed scripts and manual data alignment (Worsley, 2018 ), MMD can be inaccessible to those who are not already invested in this type of research and/or do not have the necessary technical competence.…”

Section: Practitioner Notesmentioning

confidence: 99%

Multimodal data capabilities for learning: What can multimodal data tell us about learning?

Sharma¹,

Giannakos²

2020

Brit J Educational Tech

Self Cite

119

View full text Add to dashboard Cite

Most research on learning technology uses clickstreams and questionnaires as their primary source of quantitative data. This study presents the outcomes of a systematic literature review of empirical evidence on the capabilities of multimodal data (MMD) for human learning. This paper provides an overview of what and how MMD have been used to inform learning and in what contexts. A search resulted in 42 papers that were included in the analysis. The results of the review depict the capabilities of MMD for learning and the ongoing advances and implications that emerge from the employment of MMD to capture and improve learning. In particular, we identified the six main objectives (ie, behavioral trajectories, learning outcome, learning‐task performance, teacher support, engagement and student feedback) that the MMLA research has been focusing on. We also summarize the implications derived from the reviewed articles and frame them within six thematic areas. Finally, this review stresses that future research should consider developing a framework that would enable MMD capacities to be aligned with the research and learning design (LD). These MMD capacities could also be utilized on furthering theory and practice. Our findings set a baseline to support the adoption and democratization of MMD within future learning technology research and development. What is already known about this topic Capturing and measuring learners’ engagement and behavior using MMD has been explored in recent years and exhibits great potential. There are documented challenges and opportunities associated with capturing, processing, analyzing and interpreting MMD to support human learning. MMD can provide insights into predicting learning engagement and performance as well as into supporting the process. What this paper adds Provides a systematic literature review (SLR) of empirical evidence on MMD for human learning. Summarizes the insights MMD can give us about the learning outcomes and process. Identifies challenges and opportunities of MMD to support human learning. Implications for practice and/or policy Learning analytics researchers will be able to use the SLR as a guide for future research. Learning analytics practitioners will be able to use the SLR as a summary of the current state of the field.

show abstract

Section: Practitioner Notesmentioning

confidence: 99%

Multimodal data capabilities for learning: What can multimodal data tell us about learning?

Sharma¹,

Giannakos²

2020

Brit J Educational Tech

Self Cite

119

View full text Add to dashboard Cite

show abstract

“…At present, research in MMLA has focused on studying and modelling learning strategies (Mangaroska, Sharma, Giannakos, Traeteberg, & Dillenbourg, 2018;Worsley & Blikstein, 2015), predicting high-level constructs such as learners' attention and engagement (Chan, Ochoa, & Clarke, 2020), building more accurate learner models (Giannakos, Sharma, Pappas, Kostakos, & Velloso, 2019), designing multimodal learning interfaces (Echeverria et al, 2019), or generating new insights into teaching and learning at a more fine-grained level (Martinez-Maldonado et al, 2020;Prieto, Sharma, Kidzinski, Rodríguez-Triana, & Dillenbourg, 2018). In fact, MMLA promises to bridge theory and complex learning behaviour (Worsley, 2014), support educators' role in addressing students' individual needs, expectations, and skills in physical (Ogan, 2019) and digital (Ochoa et al, 2018) learning settings, as well as tackle the complexities of orchestrating (e.g., designing, managing, adapting) learning activities at multiple social levels beyond the challenges (e.g., time, mental effort) instructors face in their everyday practice (Prieto et al, 2018).…”

Section: Multimodal Learning Analytics Researchmentioning

confidence: 99%

Multimodal Learning Analytics to Inform Learning Design: Lessons Learned from Computing Education

Mangaroska

Sharma

Gašević

et al. 2020

JLA

Self Cite

View full text Add to dashboard Cite

Programming is a complex learning activity that involves coordination of cognitive processes and affective states. These aspects are often considered individually in computing education research, demonstrating limited understanding of how and when students learn best. This issue confines researchers to contextualize evidence-driven outcomes when learning behaviour deviates from pedagogical intentions. Multimodal learning analytics (MMLA) captures data essential for measuring constructs (e.g., cognitive load, confusion) that are posited in the learning sciences as important for learning, and cannot effectively be measured solely with the use of programming process data (IDE-log data). Thus, we augmented IDE-log data with physiological data (e.g., gaze data) and participants’ facial expressions, collected during a debugging learning activity. The findings emphasize the need for learning analytics that are consequential for learning, rather than easy and convenient to collect. In that regard, our paper aims to provoke productive reflections and conversations about the potential of MMLA to expand and advance the synergy of learning analytics and learning design among the community of educators from a post-evaluation design-aware process to a permanent monitoring process of adaptation.

show abstract

“…As described in Section 3.1.2, another frequently used theory in MMLA research is CLT, which is discussed in six papers (see Figure 3). In four of these works, CLT is combined with other theories to account for other cognitive (Mangaroska et al, 2018) aspects of learning as well as the embodiment (Lee‐Cultura et al, 2022) and Papert's notion of constructionism (Papavlasopoulou et al, 2018). All papers focusing on CLT use MMLA with the goal of accessing learners' cognitive capacities (eg, how they process information; Mangaroska et al, 2018) and identifying differences and commonalities between the observed or self‐reported cognition of learners and cognitive‐related MMLA measurements that are unobservable by naked eye, such as CL based on galvanic skin responses (Larmuseau et al, 2020) and CL and arousal (Lee‐Cultura et al, 2022).…”

Section: Resultsmentioning

confidence: 99%

The role of learning theory in multimodal learning analytics

Giannakos

Cukurova

2023

Brit J Educational Tech

Self Cite

View full text Add to dashboard Cite

This study presents the outcomes of a semi‐systematic literature review on the role of learning theory in multimodal learning analytics (MMLA) research. Based on previous systematic literature reviews in MMLA and an additional new search, 35 MMLA works were identified that use theory. The results show that MMLA studies do not always discuss their findings within an established theoretical framework. Most of the theory‐driven MMLA studies are positioned in the cognitive and affective domains, and the three most frequently used theories are embodied cognition, cognitive load theory and control–value theory of achievement emotions. Often, the theories are only used to inform the study design, but there is a relationship between the most frequently used theories and the data modalities used to operationalize those theories. Although studies such as these are rare, the findings indicate that MMLA affordances can, indeed, lead to theoretical contributions to learning sciences. In this work, we discuss methods of accelerating theory‐driven MMLA research and how this acceleration can extend or even create new theoretical knowledge. Practitioner notesWhat is already known about this topic Multimodal learning analytics (MMLA) is an emerging field of research with inherent connections to advanced computational analyses of social phenomena. MMLA can help us monitor learning activity at the micro‐level and model cognitive, affective and social factors associated with learning using data from both physical and digital spaces. MMLA provide new opportunities to support students' learning. What this paper adds Some MMLA works use theory, but, overall, the role of theory is currently limited. The three theories dominating MMLA research are embodied cognition, control–value theory of achievement emotions and cognitive load theory. Most of the theory‐driven MMLA papers use theory ‘as is’ and do not consider the analytical and synthetic role of theory or aim to contribute to it. Implications for practice and/or policy If the ultimate goal of MMLA, and AI in Education in general, research is to understand and support human learning, these studies should be expected to align their findings (or not) with established relevant theories. MMLA research is mature enough to contribute to learning theory, and more research should aim to do so. MMLA researchers and practitioners, including technology designers, developers, educators and policy‐makers, can use this review as an overview of the current state of theory‐driven MMLA.

show abstract

Gaze-Driven Design Insights to Amplify Debugging Skills: A Learner-Centered Analysis Approach

Cited by 12 publications

References 67 publications

Multimodal data capabilities for learning: What can multimodal data tell us about learning?

Multimodal data capabilities for learning: What can multimodal data tell us about learning?

Multimodal Learning Analytics to Inform Learning Design: Lessons Learned from Computing Education

The role of learning theory in multimodal learning analytics

Contact Info

Product

Resources

About