Who/What Is My Teammate? Team Composition Considerations in Human–AI Teaming

McNeese, Nathan J.; Schelble, Beau G.; Canonico, Lorenzo Barberis; Demir, Mustafa

doi:10.1109/thms.2021.3086018

Cited by 41 publications

(18 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To achieve this ideal, the AI must be given the capability to diagnose such states and provide analyses and explanations that are comprehensible to the human (van den Bosch & Bronkhorst, 2018). One of the most significant, unanswered research questions in this domain is: what features of human-human teams can be extended and applied to human-AI teams (McNeese et al, 2021)?…”

Section: Human-ai Teams: Interaction To Collaborationmentioning

confidence: 99%

Understanding Human Decision Processes: Inferring Decision Strategies From Behavioral Data

Walsh

Feigh

2022

Journal of Cognitive Engineering and Decision Making

View full text Add to dashboard Cite

This work investigates a method to infer and classify decision strategies from human behavior, with the goal of improving human-agent team performance by providing AI-based decision support systems with knowledge about their human teammate. First, an experiment was designed to mimic a realistic emergency preparedness scenario in which the test participants were tasked with allocating resources into 1 of 100 possible locations based on a variety of dynamic visual heat maps. Simple participant behavioral data, such as the frequency and duration of information access, were recorded in real time for each participant. The data were examined using a partial least squares regression to identify the participants’ likely decision strategy, that is, which heat maps they relied upon the most. The behavioral data were then used to train a random forest classifier, which was shown to be highly accurate in classifying the decision strategy of new participants. This approach presents an opportunity to give AI systems the ability to accurately model the human decision-making process in real time, enabling the creation of proactive decision support systems and improving overall human-agent teaming.

show abstract

Section: Human-ai Teams: Interaction To Collaborationmentioning

confidence: 99%

Understanding Human Decision Processes: Inferring Decision Strategies From Behavioral Data

Walsh

Feigh

2022

Journal of Cognitive Engineering and Decision Making

View full text Add to dashboard Cite

show abstract

“…HATs had lower team mental model similarity, and their similarity levels were significantly more varied than human-human teams, but HAT's had greater task mental model similarity than human-human teams. This trend reveals that even though HAT's team mental models suffer, the agent teammate is capable of setting an example for their teammates, and in doing so, they enhance the team's shared understanding of the task, as posited in prior HAT research [65]. Finally, human team members perceived significantly less team cognition with agent teammates than human teammates, as shown in the independent samples t test, and this trend was reflected in the ANOVA of the three conditions.…”

Section: Quantitative Resultsmentioning

confidence: 72%

“…Each round lasted for nine minutes, and the teams worked together for 36 minutes in total within the NeoCITIES simulation. In congruence with past literature, this amount of time is adequate to develop team cognition [65,70]. Upon completing the four rounds, the experimenter instructed the team to navigate the survey to complete the post-task measures.…”

Section: Methodsmentioning

confidence: 86%

“…NeoCITIES calculates team performance using Equations 1 and 2, which have been utilized in past HAT research [65]. The variables within Equation 1 like "(end -start)" refer to the duration it took the team to complete the event successfully, from the moment the event became active, while "limit" referred to the time limit associated with that event.…”

Section: Objective Teammentioning

confidence: 99%

See 1 more Smart Citation

Let's Think Together! Assessing Shared Mental Models, Performance, and Trust in Human-Agent Teams

Schelble

Flathmann

McNeese

et al. 2022

Proc. ACM Hum.-Comput. Interact.

Self Cite

View full text Add to dashboard Cite

An emerging research agenda in Computer-Supported Cooperative Work focuses on human-agent teaming and AI agent's roles and effects in modern teamwork. In particular, one understudied key question centers around the construct of team cognition within human-agent teams. This study explores the unique nature of team dynamics in human-agent teams compared to human-human teams and the impact of team composition on perceived team cognition, team performance, and trust. In doing so, a mixed-method approach, including three team composition conditions (all human, human-human-agent, human-agent-agent), completed the team simulation NeoCITIES and completed shared mental model, trust, and perception measures. Results found that human-agent teams are similar to human-only teams in the iterative development of team cognition and the importance of communication to accelerating its development; however, human-agent teams are different in that action-related communication and explicitly shared goals are beneficial to developing team cognition. Additionally, human-agent teams trusted agent teammates less when working with only agents and no other humans, perceived less team cognition with agent teammates than human ones, and had significantly inconsistent levels of team mental model similarity when compared to human-only teams. This study contributes to Computer-Supported Cooperative Work in three significant ways: 1) advancing the existing research on human-agent teaming by shedding light on the relationship between humans and agents operating in collaborative environments, 2) characterizing team cognition development in human-agent teams; and 3) advancing real-world design recommendations that promote human-centered teaming agents and better integrate the two.

show abstract

“…Combining human and AI strengths can reduce human workload, improve situational awareness, scale military forces, reduce casualties, improve system resilience, and facilitate effective decision-making in combat scenarios. Human-agent teams (HATs) can be defined as a collaboration of one or more intelligent autonomous agents and human counterparts, each working as a full-fledged member of a team towards a common goal (O'Neill et al, 2020;McNeese et al, 2018;McNeese et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

Mutually beneficial decision making in Human-AI teams: Understanding soldier’s perception and expectations from AI teammates in human-AI teams

Sawant

Mallick

McNeese

et al. 2022

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

Who/What Is My Teammate? Team Composition Considerations in Human–AI Teaming

Cited by 41 publications

References 48 publications

Understanding Human Decision Processes: Inferring Decision Strategies From Behavioral Data

Understanding Human Decision Processes: Inferring Decision Strategies From Behavioral Data

Let's Think Together! Assessing Shared Mental Models, Performance, and Trust in Human-Agent Teams

Mutually beneficial decision making in Human-AI teams: Understanding soldier’s perception and expectations from AI teammates in human-AI teams

Contact Info

Product

Resources

About