Exploration of Team Communication Behaviors from a Live Training Event

Saville, Jason D.; Spain, Randall D.; Johnston, Joan; Lester, James C.

doi:10.1007/978-3-030-80000-0_13

Cited by 3 publications

(2 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 2 presents some example utterances along with their labels. Prior studies that used this labeled data found that high-performing squads shared information more frequently and issued more commands to teammates compared to their lower-performing counterparts (Saville et al, 2021). Additionally, high-performing teams maintained consistent rates of information exchange regardless of scenario di culty, while low-performing teams showed signi cant increases in commands and information exchange during stressful situations (Saville et al, 2022).…”

Section: Training Environment and Datasetmentioning

confidence: 85%

“…Information ow indicates the direction of information being passed (e.g., up or down the chain of command) during communication. Prior research suggests that the nuances in communication -speci cally, the frequency and type of dialogue acts and information ows-correlate strongly with team performance levels (Saville et al, 2021;. These ndings underscore the critical role of information ow in understanding team dynamics and outcomes, which exhibits signi cant potential to model together with dialogue acts through multi-task modeling.…”

Section: Baseline Modelsmentioning

confidence: 85%

See 1 more Smart Citation

Applying Large Language Models to Enhance Dialogue and Communication Analysis for Adaptive Team Training

Spain,

Min,

Kumaran

et al. 2024

Preprint

View full text Add to dashboard Cite

Adaptive training environments that can analyze team communication content and provide remediation to facilitate team coordination offer great potential for enhancing adaptive training systems for teams. Developing computational models that can perform robust team communication analytics based on small datasets is challenging. Large language models (LLMs) offer significant potential to address these challenges and enhance dialogue act classification performance using zero-shot and few-shot learning. This paper evaluates LLMs against previous state-of-the-art methods, with an emphasis on dialogue act recognition performance and error analysis for identifying frequently misclassified instances. Results from a small team communication dataset indicate that zero-shot LLMs, particularly those utilizing GPT-4 and refined through robust prompt engineering, achieve significant classification performance improvements in dialogue act recognition compared to previous state-of-the-art transformer-based models fine-tuned with team communication data. Error analysis shows that the prompt refinements, especially those aimed at clarifying confusion between dialogue acts, result in superior recall rates for challenging dialogue act labels by effectively handling complex dialogue scenarios and ambiguities within communication data. Our transformer-based framework demonstrates its effectiveness in achieving high accuracy rates in dialogue act recognition with minimal training data, underscoring its potential to enhance team training programs by providing adaptive feedback. This approach paves the way for developing AI-enabled training systems that can adapt to the dynamic communication styles of different teams.

show abstract

Section: Training Environment and Datasetmentioning

confidence: 85%

Section: Baseline Modelsmentioning

confidence: 85%