A data driven approach to assess team performance through team communication.

Abstract

For teams working in complex task environments, instilling effective communication between team members is a primary goal during task training. Presently, responsibility for evaluating team communication abilities resides with instructors and outside observers who make qualitative assessments that are shared with the team following a training exercise. Constructing technologies to automate these assessments has historically been prohibitive for two reasons. First, the financial cost of instrumenting the environment to collect team communication data at the necessary fidelity has been too high for an operational setting. Second, past research on using team communication as a proxy for team performance assessment has relied on defining communication through traditional algorithmic design, an approach which does not properly capture the varied nature of communication strategies amongst different teams. Recent scientific research in team dynamics provides a theoretical framework leading to a datadriven solution for analyzing the effectiveness of team communication. By framing team communication as an emergent data stream from a complex system, one may employ machine learning or other statistical-analysis tools to highlight communication patterns and variance, both shown as effective means for assessing team adaptability to novel scenarios. Furthermore, lowcost wearable computers (e.g., smartphones) have opened new possibilities for observing people's interactions in natural settings to better analyze and improve team performance. This report summarizes research conducted by Sandia National Laboratories in developing a data-driven approach to analyzing team communications within the context of Surfaced Piloting and Navigation (SPAN) training for submariners. Using Dynamic Bayesian Networks (DBN's), this approach created predictive models of communication patterns that emerge from the team in different contexts. Based upon data collection conducted in the lab and within live submarine crew training, our results demonstrate the robust nature of DBN's by still identifying key communication events even when teams altered their speaking patterns during these events to accommodate for novel changes in the scenario.