Towards Automatic Collaboration Analytics for Group Speech Data Using Learning Analytics

Abstract

Collaboration is an important 21st Century skill. Co-located (or face-to-face) collaboration (CC) analytics gained momentum with the advent of sensor technology. Most of these works have used the audio modality to detect the quality of CC. The CC quality can be detected from simple indicators of collaboration such as total speaking time or complex indicators like synchrony in the rise and fall of the average pitch. Most studies in the past focused on “how group members talk” (i.e., spectral, temporal features of audio like pitch) and not “what they talk”. The “what” of the conversations is more overt contrary to the “how” of the conversations. Very few studies studied “what” group members talk about, and these studies were lab based showing a representative overview of specific words as topic clusters instead of analysing the richness of the content of the conversations by understanding the linkage between these words. To overcome this, we made a starting step in this technical paper based on field trials to prototype a tool to move towards automatic collaboration analytics. We designed a technical setup to collect, process and visualize audio data automatically. The data collection took place while a board game was played among the university staff with pre-assigned roles to create awareness of the connection between learning analytics and learning design. We not only did a word-level analysis of the conversations, but also analysed the richness of these conversations by visualizing the strength of the linkage between these words and phrases interactively. In this visualization, we used a network graph to visualize turn taking exchange between different roles along with the word-level and phrase-level analysis. We also used centrality measures to understand the network graph further based on how much words have hold over the network of words and how influential are certain words. Finally, we found that this approach had certain limitations in terms of automation in speaker diarization (i.e., who spoke when) and text data pre-processing. Therefore, we concluded that even though the technical setup was partially automated, it is a way forward to understand the richness of the conversations between different roles and makes a significant step towards automatic collaboration analytics.

Literature Review on Co-Located Collaboration Modeling Using Multimodal Learning Analytics—Can We Go the Whole Nine Yards?

“…Most of the works in the field of learning analytics support for collaboration have focused on the analysis of distributed (or online) collaboration [4]. However, with the pervasive use of sensors [5], [6], multimodal learning analytics (MMLA) [7]- [9] has picked up the pace, thus shifting the focus to the analysis of colocated collaboration (CC) (or face-to-face collaboration) with the help of sensor technology [5], [6], [10], [11]. Furthermore, sensor technology can be easily scaled up [12] and has become affordable and reliable in the past decade [13].…”

Multimodal large language models for inclusive collaboration learning tasks

“…The inclusive collaboration rubric pulls from existing research on collaboration quality that identifies four collaboration indicators (information sharing, reciprocal interaction, shared understanding, and inclusion) from participants' audio, text, and video data (Cukurova et al, 2018;Praharaj et al, 2021), and the technical feat of capturing and preprocessing collaborative exchanges is informed by previous scholarship in Multimodal Learning Analytics research (Ochoa et al, 2013;Worsley and Blikstein, 2015). Automatic distillation of raw data into collaboration features would include: automatic speech recognition, computational linguistic methods to clean, parse, and analyze transcribed dialogue (eg.…”

“…The contributions of Phase 1 are multiple: to expand beyond research that analyzes collaborative language at the surface level, such as looking at word counts or temporal durations, and support deeper content-level analysis (Praharaj et al, 2021); to map current trends in large language modeling to theoretically-sound learning and inclusion frame-works that extend past pure performance measures and support responsible downstream usage of such models.…”

“…One involves using the large language model to provide a nuanced view of collaborative linguistic exchanges at the content level. According to Praharaj et al (2021) note that very few studies integrate an analysis of "verbal audio indicators or the content of the audio for the analysis of [in-person] collaboration quality" (pg. 2).…”

On the Use of Large Interactive Displays to Support Collaborative Engagement and Visual Exploratory Tasks

“…In collaborative learning, individuals can share information and ideas they explore and discover with others in the group. Computer-supported collaborative learning (CSCL) extends traditional cooperative learning through a technology-enhanced collaborative learning environment, platform, or medium [18][19][20]. Research in CSCL include various types of technologies such as mobile tablets [21,22], large tabletop or wall displays [23], and more recently virtual reality (VR) and augmented reality (AR) [24,25].…”