Going beyond what is visible: What multichannel data can reveal about interaction in the context of collaborative learning?

Malmberg, Jonna; Järvelä, Sanna; Holappa, Jukka; Haataja, Eetu; Huang, Xiaohua; Siipo, Antti

doi:10.1016/j.chb.2018.06.030

Cited by 87 publications

(93 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The availability and minimal intrusiveness of video cameras (compact cameras that may be smaller than a coin or incorporated in any mobile device), in conjunction with the potential of machine learning, makes them a device that is commonly used for studying aspects of understanding, achievement, emotions, attention and speech in MMLA research with children. Children’s actions can be recorded in a wide‐angle shot (Pérez, Martínez, Avila & Espinosa, 2018; Ramakrishnan et al ., 2019), with a 360° point‐of‐view multi‐camera system (Malmberg et al ., 2019) or with a webcam on top of a computer screen (Chen et al ., 2016; Pereira et al ., 2018; Sharma et al ., 2019) to analyse the emotions generated during the learning process automatically. Spikol and colleagues (2018) calculated the distance between the participants (recorded with a front camera), which they treated as a proxy for collaboration.…”

Section: Resultsmentioning

confidence: 99%

“…In some studies, these sensors were connected to a laptop or mini‐PC using cables (Rahman & Bhuiyan, 2015 proposes keeping this hardware in a cartridge case). Wireless wristbands are more usual and allow the data related to heart rate, electrodermal activity and temperature to be collected (Goodwin, Mazefsky, Ioannidis, Erdogmus & Siegel, 2019; Kourakli et al ., 2017; Malmberg et al ., 2019; Noroozi et al ., 2019). Rudovic et al .…”

Section: Resultsmentioning

confidence: 99%

“…(2018) is an example of “supervised” MMLA, in which LA are used alongside five “human experts” who evaluate the student’s engagement and affect in the recorded video; the evaluations are then aligned with the LA. This is not an isolated case: often it is the human coders who analyse the videos of game sessions and the researchers then compare and validate or complement their evaluations with the results of the processed LA (Malmberg et al ., 2019; Martinez‐Maldonado et al ., 2017; Spikol et al ., 2018; Worsley, 2018). In this sense, quantitative data coded by human researchers through structured video observation are part of the MMLA dataset (Pérez et al ., 2018; Worsley, 2018).…”

Section: Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Multimodal Learning Analytics research with young children: A systematic review

Lanna¹

2020

Brit J Educational Tech

View full text Add to dashboard Cite

Learning Analytics and Multimodal Learning Analytics are changing the way of analysing the learning process while students interact with an educational content. This paper presents a systematic literature review aimed at describing practices in recent Multimodal Learning Analytics and Learning Analytics research literature in order to identify tools and strategies useful for the assessment of the progress and behaviour of children under 6 years old in respect of their learning. The purpose is to provide guidance for Multimodal Learning Analytics research with children under 6 years old to assess their engagement in a task, their emotions, attention, understanding and achievement of a goal. The current state of knowledge on Multimodal Learning Analytics research suggests how performance analytics, face and speech recognition systems, eye tracking, Kinect analytics and wristbands could be used with children. The results show the complexity of collecting data using non‐invasive methodologies with children under 6 years old. Ethical implications related to multimodal data from audio, visual, biometric and quantitative measures of child behaviour are discussed.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal Learning Analytics research with young children: A systematic review

Lanna¹

2020

Brit J Educational Tech

View full text Add to dashboard Cite

show abstract

“…Currently, we are progressing in our analysis of different motivational and emotional indicators that contribute to the process of regulated learning and provide evidence on how motivation and emotion regulation plays a role in group interaction during collaborative learning (Järvenoja et al, 2015;Malmberg et al, 2017;Malmberg, Järvelä, Holappa, Haataja, Siipo, & Huang, 2018). At the moment, we are exploring how the research on motivation and emotion regulation in collaborative learning could benefit from implementation of physiological measures.…”

Section: Claim: Motivation and Emotion Regulation Is Intertwined Withmentioning

confidence: 99%

Capturing motivation and emotion regulation during a learning process

Järvenoja

Järvelä

Törmänen

et al. 2018

FLR

Self Cite

View full text Add to dashboard Cite

This paper describes our research approach in which we have focused on situational and contextual variations in motivation and emotion regulation to better understand its role, appearance and function in collaborative learning situations. We have used research designs that employ process-oriented measures combined with subjective interpretations to capture motivation and emotion regulation. Analysing on-line process data poses several challenges such as variation in the granularity of different data sources, problems that emerge due to the complexity of contextual and situational factors in ecologically-valid learning situations or, currently, challenges in the use of multiple data channels and their analyses.In this paper, we present three claims underlying our research, particularly the motivation and emotions and their regulation in learning. The claims are as follows: (1) motivation and emotion regulation is situation and context specific, (2) motivation and emotion regulation is influenced by multi-layered nature of motivation and (3) Motivation and emotion regulation is intertwined with other processes of learning and can be captured from their temporal manifestation. We present an example from our empirical study to discuss how these claims have led us to employ multiple process-oriented methods that include both subjective and objective data sources, including different combinations of situation-specific self-reports, video and physiological data. We then describe opportunities and challenges involved in the empirical studies.

show abstract

“…For example, based on the correct prediction of an image, the computer vision system can automatically select the candidate photos for people to make the photo album [12]. This kind of system may also assist social scientists/researchers in the field of education to analyze the interaction of students in collaborative learning [13], etc. Therefore, in this paper we mainly focus on analyzing the basic emotions exhibited by a group of people in an image, namely, group-level emotion.…”

Section: Introductionmentioning

confidence: 99%

Analyzing Group-Level Emotion with Global Alignment Kernel based Approach

Huang

Dhall

Goecke

et al. 2022

IEEE Trans. Affective Comput.

Self Cite

View full text Add to dashboard Cite

From the perspective of social science, understanding group emotion has become increasingly important for teams to considerably accomplish organizational work. Currently, automatically analyzing the perceived affect of a group of people has been received increasingly interest in affective computing community. The variability in group size makes difficulty for group-level emotion recognition to straightforwardly measure the feature distance of two group-level images. Recent works attempted to resolve the preceding problem by using feature encoding. However, the early works lack of efficiency. To alleviate this problem, this paper aims to design a new method to effectively analyze the group behavior from a group-level image. Motivated by time-series kernel approaches explored in dynamic facial expression classification, this paper mainly concentrates on global alignment kernel and design support vector machine with the combined global alignment kernels (SVM-CGAK) to better recognize group-level emotion. Specifically, we first propose to use global alignment kernel to explicitly measure the distance of two group-level images. For improving the performance of global alignment kernel, we use the global weight sort scheme based on their spatial relation information to sort the faces from group-level image, making an efficient data structure to the global alignment kernel. With this new global alignment kernel, we construct the backbone of SVM-CGAK, namely, support vector machine with global alignment kernel. Furthermore, considering the challenging environment, we construct two global alignment kernels based on Reisz-based Volume Local Binary Pattern and deep convolutional neural network features, respectively. Lastly, to make the robustness of group-level emotion recognition, we propose SVM-CGAK combining both global alignment kernels with multiple kernel learning approach. It can enhance the discriminative ability of each global alignment kernel. Intensive experiments are conducted on three challenging group-level emotion databases. The experimental results demonstrate that the proposed approach achieves promising performance for group-level emotion recognition compared with the recent state-of-the-art methods.

show abstract

Going beyond what is visible: What multichannel data can reveal about interaction in the context of collaborative learning?

Cited by 87 publications

References 54 publications

Multimodal Learning Analytics research with young children: A systematic review

Multimodal Learning Analytics research with young children: A systematic review

Capturing motivation and emotion regulation during a learning process

Analyzing Group-Level Emotion with Global Alignment Kernel based Approach

Contact Info

Product

Resources

About