Assessing multi-agent human-autonomy teams: US Army Robotic Wingman gunnery operations

Schaefer, Kristin E.; Baker, Anthony L.; Brewer, Ralph W.; Patton, Debra J.; Canady, Jonroy; Metcalfe, J. Stanley

doi:10.1117/12.2519302

Cited by 12 publications

(4 citation statements)

References 77 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although IAs will continue to advance and acquire more capabilities, the goal of using these systems is not to replace the human, or to provide more "complex tools" for humans to use, but rather to effectively "team" autonomy with humans so they achieve synergy and are able to work cooperatively in complex, dynamic environments (Phillips et al 2011;Johnson and Vera 2019). For example, the US military is researching how human-autonomy teams can maintain overmatch in unpredictable adversarial environments including intelligence, surveillance, and reconnaissance (Harris and Barber 2014;Kattoju et al 2016) as well as lethality (Brewer et al 2018;Schaefer et al 2019a) while reducing the cognitive burden on Soldiers. A significant challenge is integrating humans and autonomous systems into interdependent, heterogeneous teams that are able to achieve appropriate levels of team trust and cohesion.…”

Section: Defining Human-autonomy Teamsmentioning

confidence: 99%

“…When integrating autonomy into the team, it is important to understand that human-autonomy trust functions differently from trust between humans (Baker et al 2018), but well-calibrated trust in autonomy is still positively correlated with team performance (McNeese et al 2019). To understand and contextualize trust and performance in a human-autonomy team, then, it is critical to evaluate the communication of the team, as this provides a window into the team's coordination, information sharing, decision-making, and more (Baker et al 2019;Baker et al 2020a, b;Schaefer et al 2019a).…”

Section: Team Trust and Cohesionmentioning

confidence: 99%

“…The current implementation of the model has been trained on an existing speech dataset called the IEMOCAP dataset (Busso et al 2008). Current efforts include the development of a labeled dataset from recent US Army field studies using the Wingman human-autonomy lethality platform in a gunnery task (Schaefer et al 2019a;Schaefer et al 2019b).…”

Section: Vocal Feature Assessment Using Neural Networkmentioning

confidence: 99%

See 2 more Smart Citations

Approaches for assessing communication in human-autonomy teams

Baker

Fitzhugh

Huang

et al. 2021

Hum.-Intell. Syst. Integr.

View full text Add to dashboard Cite

Evaluation of team communication can provide critical insights into team dynamics, cohesion, trust, and performance on joint tasks. Although many communication-based measures have been tested and validated for human teams, this review article extends this research by identifying key approaches specific to human-autonomy teams. It is not possible to identify all approaches for all situations, though the following seem to generalize and support multi-size teams and a variety of military operations. Therefore, this article will outline several key approaches to assessing communication, associated data requirements, example applications, verification of methods through HAT use cases, and lessons learned, where applicable. Some approaches are based on the structure of team communication; others draw from dynamical systems theory to consider perspectives across different timescales; other approaches leverage features of team members’ voices or facial expressions to detect emotional states that can provide windows into other workings of the team; still others consider the content of communication to produce insights. Taken together, these approaches comprise a varied toolkit for deriving critical information about how team interactions affect, and are affected by, coordination, trust, cohesion, and performance outcomes. Future research directions describe four critical areas for further study of communication in human-autonomy teams.

show abstract

Section: Defining Human-autonomy Teamsmentioning

confidence: 99%

Section: Team Trust and Cohesionmentioning

confidence: 99%

Section: Vocal Feature Assessment Using Neural Networkmentioning

confidence: 99%

See 1 more Smart Citation

Approaches for assessing communication in human-autonomy teams

Baker

Fitzhugh

Huang

et al. 2021

Hum.-Intell. Syst. Integr.

View full text Add to dashboard Cite

show abstract

“…Wingman Joint Capabilities Technology Demonstration (JCTD) field experiments were conducted in 2018 and 2019 to assess the capabilities of manned-unmanned vehicle teams on standardized Army gunnery evaluation tasks (for more information about the field experiments, see Brewer et al 2018;Schaefer et al 2019aSchaefer et al , 2019bBaker et al 2020). In these field experiments, Soldiers and Marines crewed a standard Army vehicle that was adapted to support interaction with an unmanned robotic combat vehicle to perform a series of live-fire target engagements.…”

Section: Thementioning

confidence: 99%

Communication Metrics for Human-Autonomy Teaming: Lessons Learned from us Army Gunnery Field Experiments

Baker

Fitzhugh

Forster

et al. 2021

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

The development of more effective human-autonomy teaming (HAT) will depend on the availability of validated measures of their performance. Communication provides a critical window into a team’s interactions, states, and performance, but much remains to be learned about how to successfully carry over communication measures from the human teaming context to the HAT context. Therefore, the purpose of this paper is to discuss the implementation of three communication assessment methodologies used for two Wingman Joint Capabilities Technology Demonstration field experiments. These field experiments involved Soldiers and Marines maneuvering vehicles and engaging in live-fire target gunnery, all with the assistance of intelligent autonomous systems. Crew communication data were analyzed using aggregate communication flow, relational event models, and linguistic similarity. We discuss how the assessments were implemented, what they revealed about the teaming between humans and autonomy, and lessons learned for future implementation of communication measurement approaches in the HAT context.

show abstract