Validating team communication data using a transmission-duration threshold and voice activity detection algorithm

Abstract

The processes underlying team effectiveness can be understood by analyzing the temporal dynamics of team communication sequences. The results of such analyses have shown that the complexity of team communication is associated with team performance on task-related variables, and hence communication complexity statistics have been proposed for use as measures for real-time feedback on team performance. In two analyses of historical team communication sequences, we found that filtering via use of a transmission-duration threshold and voice activity detection algorithm resulted in significant changes in complexity relative to not filtering the data or using a transmission-duration filter alone. The use of these filtering techniques showed significant effects on the complexity of communication sequences in both a laboratory-based experiment, with participants with little experience with voice communication protocols, and in a mission simulation with trained military operators. There was also a significant non-linear relationship between the complexity of communication sequences and task performance. However, an analysis of the impact of the changes in communication dynamics gained through filtering did not demonstrate that the changed temporal dynamics of filtered data better explained team performance. It is concluded that prefiltering of invalid communication data should be included during the data cleaning stage of statistical analysis as a matter of good scientific practice. Furthermore, such use of filtering will ensure that inferences made about the relationship between the complexity of communication between team members and their performance are not confounded by the presence of invalid communication events.

Evaluating the Classification Performance of Natural Language Processing-Driven Team Communication Analysis Models

“…These information exchange and coordination behaviors allow teams to perform well in demanding situations. By investigating teams’ speech-based communication, researchers have been able to identify breakdowns in taskwork, determine periods of high workload, and develop metrics for assessing team coordination, team cognition, and team cohesion (Butchibabu et al, 2016; Grimm et al, 2022; Entin & Serfaty 1999; Hosking et al, 2019).…”