Skills Matter: Modeling the relationship between decision making processes and collaborative problem-solving skills during Hidden Profile Tasks

Lin, Yen-Hui; Dowell, Nia; Godfrey, Andrew

doi:10.1145/3448139.3448180

Cited by 2 publications

(1 citation statement)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Next, several researchers indicate that the research focus should go beyond monitoring low‐level behaviours and focus on higher‐level constructs to monitor quality of collaboration (Stewart et al, 2019). One avenue for doing so is the use of verbal and non‐verbal data to investigate natural language communication (eg, Lin et al, 2021; Praharaj et al, 2022). However, examples of studies taking into account verbal and non‐verbal data are limited (eg, Stewart et al, 2021), and so is the research on the use of LIWC.…”

Section: Learning Analytics For Cpsmentioning

confidence: 99%

Assessing verbal interaction of adult learners in computer‐supported collaborative problem solving

Buseyne,

Rajagopal,

Danquigny

et al. 2023

Brit J Educational Tech

View full text Add to dashboard Cite

The objective of this study is to explore new ways of assessing collaborative problem solving (CPS) processes based on different modalities of audio data and their combination. The data collection took place in an educational lab setting during an experiment with adult teams from professional contexts who collaboratively solved multiple problems as part of a CPS training. From audio data, both verbal (ie, speech) and non‐verbal (ie, pitch) aspects were extracted. Four analysis methods were used, including (a) content analysis; (b) linguistic inquiry and word count; (c) verbal entrainment analysis; and (d) acoustic–prosodic entrainment based on pitch data. Insights are given into the CPS processes of the participating groups using these measures and relevant relationships between some of these measures are further investigated. Based on content analysis, it was found that most of the interactions during the CPS process are task oriented, whereas team‐oriented interactions are less present. Second, three measures of proportion of contribution in CPS were investigated and clear differences in participation patterns between and within teams were found. We suggest that a combination of utterance count and words per sentence could provide valuable insights for quantity and equality of participation. Third, the study explored pronoun use and found that the most frequently used personal pronouns were first‐person singular. Next, the results indicated a relationship between pronoun use and the relative frequency of interactions. Fourth, a rather weak relationship between lexical entrainment measures and the acoustic–prosodic measures were found, suggesting that these measures are indicative of separate communicative aspects in CPS. This study contributes to a better understanding of which type of audio‐based data is most informative to teachers and students as a feedback or assessment tool. This study complements previous research as it focuses on spoken human‐to‐human communication collected in an authentic context. Practitioner notesWhat is already known about this topic Support and guidance systems for learning coaches, teachers and learners are needed to foster the educational quality of collaborative problem solving (CPS) activities. CPS is a complex process and measuring the quality of CPS processes remains challenging. Multimodal learning analytics, focusing on verbal and non‐verbal data sources and using content analysis, linguistic inquiry and word count and verbal and acoustic entrainment measures could be valuable to measure the quality of CPS. What this paper adds The majority of interactions during CPS processes are task oriented or cognitive of nature, whereas team‐oriented interactions are less present. Utterance count and words per sentence should be used in combination, as they are indicative of different aspects. Pronoun use in learners' discourse is related to the types of CPS interactions. Lexical entrainment measures and acoustic–prosodic are indicative of distinctive communicative aspects in CPS. Implications for practice and/or policy Quality indicators of CPS processes should include both verbal and non‐verbal measures of students' interactions. Educational researchers and the (Edtech) industry should further leverage their forces to foster the development of (semi‐)automated systems for measuring the quality of CPS processes. It should be further investigated how quality indicators of CPS processes can be most meaningful to trainers, teachers and learners, for example, through the use of dashboards.

show abstract