Decentralized Refinement Planning and Acting

Abstract

We describe Dec-RPAE, a system for decentralized multi-agent acting and planning in partially observable and non-deterministic environments. The system includes both an acting component and an online planning component. The acting component is similar to RAE, a well-known acting engine, but incorporates changes that enable it to be used by multiple autonomous agents working independently in a collaborative setting. Each agent runs a local copy of Dec-RPAE, with a set of hierarchical refinement methods using operational models that specify various ways to accomplish its designated tasks. To perform actions, the agent uses Dec-RPAE’s acting component to execute the methods in the agent’s environment. To advise the acting component on which method to execute, the planning component repeatedly does Monte Carlo simulations of the methods to estimate their potential outcomes. Agents can communicate with each other to exchange information about their states, tasks, goals, and plans in order to cooperatively succeed in their respective missions. Our experimental results demonstrate that Dec-RPAE is useful for improving the agents’ performances.

Collaborative Cost Multi-Agent Decision-Making Algorithm with Factored-Value Monte Carlo Tree Search and Max-Plus

“…The subject of multi-agent decision-making behavior coordination is a popular research topic that is tackled in various communities such as Game Theory, Reinforcement Learning, Decision Theory, Cybersecurity, Constraint Programming, Control, and Robotics, to mention a few [11,[13][14][15][16][17][18][19][20]. The incorporation of game theory principles into algorithms facilitates the implementation of advanced and strategic decision-making processes inside intricate contexts.…”

Flexible FOND HTN Planning: A Complexity Analysis

“…In particular when facing real-world problems, we may face challenges and limitations if we model deterministically. The world may be dynamically changing (Patra et al 2020(Patra et al , 2021Li, Patra, and Nau 2021), partially observable (Richter and Biundo 2017), or require reasoning over actions with nondeterministic outcomes (Kuter and Nau 2004;Kuter et al 2005Kuter et al , 2009) -most of these works in the realm of HTN planning and uncertainty focused on developing planners that produce classical policies to (non-hierarchical) fully observable nondeterministic (FOND) problems.…”