Impact of Heterogeneity and Risk Aversion on Task Allocation in Multi-Agent Teams

Abstract

Cooperative multi-agent decision-making is a ubiquitous problem with many real-world applications. In many practical applications, it is desirable to design a multi-agent team with a heterogeneous composition where the agents can have different capabilities and levels of risk tolerance to address diverse requirements. While heterogeneity in multi-agent teams offers benefits, new challenges arise including how to find optimal heterogeneous team compositions and how to dynamically distribute tasks among agents in complex operations. In this work, we develop an artificial intelligence framework for multi-agent heterogeneous teams to dynamically learn task distributions among agents through reinforcement learning. The framework extends Decentralized Partially Observable Markov Decision Processes (Dec-POMDP) to be compatible to model various types of heterogeneity. We demonstrate our approach with a benchmark problem on a disaster relief scenario. The effect of heterogeneity and risk aversion in agent capabilities and decision-making strategies on the performance of multi-agent teams in uncertain environments is analyzed. Results show that a well-designed heterogeneous team outperforms its homogeneous counterpart and possesses higher adaptivity in uncertain environments.

Task Allocation with Load Management in Multi-Agent Teams

“…The decision-making process for a team of heterogeneous agents with load management inherits the formulation of our previous work on heterogeneous teaming decentralized partially observable markov decision process (HT Dec-POMDP) [8] and incorporate two additional features allowing intelligent load management: 1) agent's option to idle, and 2) penalty on unnecessary task assignment depending on capabilities and current task assignment. The proposed framework heterogeneous teaming with load management (HTLM Dec-POMDP) is defined as a tuple ( Ḡ, α, S, Ā, T , R, Z, O, M, I, h, b 0 , γ) where:…”

“…where M i represents the state information collected by agent α i • I = P r(m|s , a, z) is the information probability • h is the planning time horizon • b 0 is the initial belief • γ is the discount factor This formulation follows the fundamental modeling of environment dynamics in HT Dec-POMDP [8], and explicitly defines various types of attributes (task-specific, sensing, and communication capabilities) for agents and the corresponding probabilities that describe the stochastic environment and the agent's perception of the environment and information. In many studies, agents stay active throughout the operations, which might lead to excessive usage and risk exposure.…”

“…In a team of collaborative agents, instead of being managed by a centralized system, decentralized agents have equal authorities in task planning and execution and ensuring operation effectiveness, with better team resilience in unexpected situations and scalability. With the help of emerging computing systems, recent advances in multi-agent task allocation [1]- [3] have deliberated on distributed systems and deep reinforcement learning methods [4]- [8] for complex scenarios in addition to centralized approaches [9]- [11]. Most coordination and collaboration strategies in efficient task allocation [12] define the objective as accomplishing tasks [8] or reaching task locations [7], [11] in fewer steps.…”

“…With the help of emerging computing systems, recent advances in multi-agent task allocation [1]- [3] have deliberated on distributed systems and deep reinforcement learning methods [4]- [8] for complex scenarios in addition to centralized approaches [9]- [11]. Most coordination and collaboration strategies in efficient task allocation [12] define the objective as accomplishing tasks [8] or reaching task locations [7], [11] in fewer steps. Although these approaches could sufficiently accomplish the problem objective, there is a lack of consideration of risk management since unanticipated situations might require additional strategic amendments in real-world operations.…”

“…In this paper, we leverage our previous work [8] on task allocation for heterogeneous multi-agent teams to model task load management based on agent preference in decisionmaking as a secondary objective along with maximizing team performance. Multi-agent teams with a reduced load respond more reliably in emergencies that require extra resources, or in situations where one of the team members is compromised [24].…”