Multimodal analytics to study collaborative problem solving in pair programming

Grover, Shuchi; Bienkowski, Marie; Tamrakar, Amir; Siddiquie, Behjat; Salter, David A.; Divakaran, Ajay

doi:10.1145/2883851.2883877

Cited by 41 publications

(42 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our case, these features highly correlate with the quality of the students' artefacts in project‐based learning. These results align with existing research on PBL activities that show the value of nonverbal indexes of student interaction in estimating their success at learning processes (Cukurova, Luckin, Millan, & Mavrikis, ) as well as MMLA research findings that show the potential of hand motion and speed, and the location of the learners to predict student success at various learning outcomes (Blikstein, ; Ochoa et al, ; Grover et al, ). As mentioned in Section 2, there are three main aspects of PBL: students are asking driving questions, doing investigations to answer these questions, and collaborate together to solve these questions (Krajcik, ).…”

Section: Discussionsupporting

confidence: 88%

“…Moreover, Ochoa and colleagues' () work used existing multimedia processing technologies to produce a set of features for accurate predictions of experts in groups of students solving math problems, which illustrated the benefits of MMLA to support students' learning in these contexts. Similarly, Chen and colleagues () expanded from the oral presentation quality data corpus to further examine the feasibility of using multimodal technologies for the assessing of public speaking skills; Grover and colleagues () explored how to develop computational models of social learning environments. In their work, Grover and colleagues () managed to classify the quality of collaboration from body movement and gestures of pair programmers working together with acceptable accuracy rates.…”

Section: Introductionmentioning

confidence: 99%

“…Similarly, Chen and colleagues () expanded from the oral presentation quality data corpus to further examine the feasibility of using multimodal technologies for the assessing of public speaking skills; Grover and colleagues () explored how to develop computational models of social learning environments. In their work, Grover and colleagues () managed to classify the quality of collaboration from body movement and gestures of pair programmers working together with acceptable accuracy rates. Although most of the existing MMLA research approaches focus on learners' data, Prieto and colleagues () and Martinez‐Maldonado and colleagues () focused their research efforts on how MMLA can support teaching actions and orchestration in the classroom.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Spikol

Ruffaldi

Dabisias

et al. 2018

Computer Assisted Learning

140

107

View full text Add to dashboard Cite

Multimodal learning analytics provides researchers new tools and techniques to capture different types of data from complex learning activities in dynamic learning environments. This paper investigates the use of diverse sensors, including computer vision, user‐generated content, and data from the learning objects (physical computing components), to record high‐fidelity synchronised multimodal recordings of small groups of learners interacting. We processed and extracted different aspects of the students' interactions to answer the following question: Which features of student group work are good predictors of team success in open‐ended tasks with physical computing? To answer this question, we have explored different supervised machine learning approaches (traditional and deep learning techniques) to analyse the data coming from multiple sources. The results illustrate that state‐of‐the‐art computational techniques can be used to generate insights into the "black box" of learning in students' project‐based activities. The features identified from the analysis show that distance between learners' hands and faces is a strong predictor of students' artefact quality, which can indicate the value of student collaboration. Our research shows that new and promising approaches such as neural networks, and more traditional regression approaches can both be used to classify multimodal learning analytics data, and both have advantages and disadvantages depending on the research questions and contexts being investigated. The work presented here is a significant contribution towards developing techniques to automatically identify the key aspects of students success in project‐based learning environments, and to ultimately help teachers provide appropriate and timely support to students in these fundamental aspects.

show abstract

Section: Discussionsupporting

confidence: 88%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Spikol

Ruffaldi

Dabisias

et al. 2018

Computer Assisted Learning

140

107

View full text Add to dashboard Cite

show abstract

“…A significant amount of work has been done using a combination of data streams to predict collaborative processes or learning outcomes (Stewart, Keirn, & D'Mello, 2018;Zhang et al, 2019). For the collaborative process, researchers have used data streams such as audio, video, logs, Kinect and physiological measures to predict features of the collaboration including coordination, dialog acts and idea generation (Ezen-Can, Grafsgaard, Lester, & Boyer, 2015;Furuichi & Worsley, 2018;Grafsgaard, Duran, Randall, Tao, & D'Mello, 2018;Grover et al, 2016;Liu & Stamper, 2017;Malmberg et al, 2019;Stewart et al, 2018;Viswanathan & VanLehn, 2017a;Worsley, 2018). In terms of learning, audio, video and logs have been used both to predict learning gains (Schneider & Blikstein, 2015) and for model refinement (Liu, Davenport, & Stamper, 2016).…”

Section: Practitioner Notesmentioning

confidence: 99%

Temporal analysis of multimodal data to predict collaborative learning outcomes

Olsen¹,

Sharma²,

Rummel³

et al. 2020

Brit J Educational Tech

View full text Add to dashboard Cite

The analysis of multiple data streams is a long‐standing practice within educational research. Both multimodal data analysis and temporal analysis have been applied successfully, but in the area of collaborative learning, very few studies have investigated specific advantages of multiple modalities versus a single modality, especially combined with temporal analysis. In this paper, we investigate how both the use of multimodal data and moving from averages and counts to temporal aspects in a collaborative setting provides a better prediction of learning gains. To address these questions, we analyze multimodal data collected from 25 9–11‐year‐old dyads using a fractions intelligent tutoring system. Assessing the relation of dual gaze, tutor log, audio and dialog data to students' learning gains, we find that a combination of modalities, especially those at a smaller time scale, such as gaze and audio, provides a more accurate prediction of learning gains than models with a single modality. Our work contributes to the understanding of how analyzing multimodal data in temporal manner provides additional information around the collaborative learning process.

show abstract

“…A classical pre/posttest approach in a controlled setting would not provide an authentic assessment of collaborative engagement. Rather, it must be assessed as development in progress [11].…”

Section: Background 21 Learner Engagementmentioning

confidence: 99%

An exploratory approach to measuring collaborative engagement in child robot interaction

Kim

Butail

Tscholl

et al. 2020

Proceedings of the Tenth International Conference on Learning Analytics &Amp; Knowledge

View full text Add to dashboard Cite

This study explored data analytic approaches to assessing young children's engagement in robot-mediated collaborative interaction. To develop our analytic models, we took a case-study approach and looked closely into four children's behaviors during three conversational sessions. Grounded in engagement theory, three sources of multimodal behavioral data (utterances, kinesics, and vocie) were coded through human annotation and automatic speech recognition and analysis. Then, information-theoretic methods were used to uncover nonlinear dependencies (called mutual information) among the multimodal behaviors of each child. From this, we derived a model to compute a compound variable of engagement. This computation produced engagement trends of each child, the engagement relationship between two children in a pair, and the engagement relationship with the robot over time. The computed trends corresponded well with the data from human observations. This approach has implications for quantifying engagement from rich and natural multimodal behaviors.

show abstract

Multimodal analytics to study collaborative problem solving in pair programming

Cited by 41 publications

References 2 publications

Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Temporal analysis of multimodal data to predict collaborative learning outcomes

An exploratory approach to measuring collaborative engagement in child robot interaction

Contact Info

Product

Resources

About