Learning Behavior Trees for Autonomous Agents with Hybrid Constraints Evolution

Zhang, Qi; Yao, Jian; Yin, Qinye; Zha, Yabing

doi:10.3390/app8071077

Cited by 23 publications

(15 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, in robotics several approaches have been proposed for learning primitive actions, e.g., [96,64,45,17], usually relying on Reinforcement Learning (RL), possibly supervised and/or with inverse RL (see survey [65]). Other techniques for learning actions as low-level skills may also be relevant, e.g., [13,124]. These and similar techniques would provide operational models of primitive actions needed by RAE, as well as a domain simulator needed by UPOM for the function Sample (see line 7 of Algorithm 6).…”

Section: Learning Operational Modelsmentioning

confidence: 99%

Deliberative acting, planning and learning with hierarchical operational models

Patra

Mason²,

Ghallab³

et al. 2021

Artificial Intelligence

View full text Add to dashboard Cite

In AI research, synthesizing a plan of action has typically used descriptive models of the actions that abstractly specify what might happen as a result of an action, and are tailored for efficiently computing state transitions. However, executing the planned actions has needed operational models, in which rich computational control structures and closed-loop online decision-making are used to specify how to perform an action in a nondeterministic execution context, react to events and adapt to an unfolding situation. Deliberative actors, which integrate acting and planning, have typically needed to use both of these models together-which causes problems when attempting to develop the different models, verify their consistency, and smoothly interleave acting and planning.As an alternative, we define and implement an integrated acting-and-planning system in which both planning and acting use the same operational models. These rely on hierarchical task-oriented refinement methods offering rich control structures. The acting component, called Reactive Acting Engine (RAE), is inspired by the well-known PRS system. At each decision step, RAE can get advice from a planner for a near-optimal choice with respect to an utility function. The anytime planner uses a UCT-like Monte Carlo Tree Search procedure, called UPOM, whose rollouts are simulations of the actor's operational models. We also present learning strategies for use with RAE and UPOM that acquire, from online acting experiences and/or simulated planning results, a mapping from decision contexts to method instances as well as a heuristic function to guide UPOM. We demonstrate the asymptotic convergence of UPOM towards optimal methods in static domains, and show experimentally that UPOM and the learning strategies significantly improve the acting efficiency and robustness.

show abstract

Section: Learning Operational Modelsmentioning

confidence: 99%

Deliberative acting, planning and learning with hierarchical operational models

Patra

Mason²,

Ghallab³

et al. 2021

Artificial Intelligence

View full text Add to dashboard Cite

show abstract

“…A second advance is to use machine learning techniques to evolve high-performing BTs for environments that can be simulated (Banerjee, 2018;Colledanchise, Parasuraman, &Ögren, 2019;Zhang, Yao, Yin, & Zha, 2018). Variants include safe learning algorithms that avoid potentially harmful states during training, e.g., by restricting controls to those that avoid disallowed states (Sprague &Ögren, 2018; for related work on safe navigation of traffic circles by autonomous vehicles, see Konda, Squires, Pierpaoli, Egerstedt, & Coogan, 2019).…”

Section: Behavior Trees (Bts) Enable Quick Responses To Unexpected Evmentioning

confidence: 99%

“…Related work on composing sequences of behaviors for multiple agents while allowing needed communication among them enables teams of agents to share information, organize themselves, and coordinate to execute complex tasks such as searching urban environments to locate and rescue victims who need help (Pierpaoli et al, 2019). Finally, combining machine learning techniques such as RL (Banerjee, 2018;Dey & Child, 2013), evolutionary programming, and deep learning (Sprague &Ögren, 2018;Zhang et al, 2018) with BTs enables agents to add to their capacities and skills over time by learning to perform new tasks, to perform old ones more efficiently, and to adapt their behaviors to new situations by modifying and adding to existing BTs or by learning new ones (Colledanchise et al, 2019).…”

Section: Behavior Trees (Bts) Enable Quick Responses To Unexpected Evmentioning

confidence: 99%

Answerable and Unanswerable Questions in Risk Analysis with Open‐World Novelty

Cox

2020

Risk Analysis

View full text Add to dashboard Cite

Decision analysis and risk analysis have grown up around a set of organizing questions: what might go wrong, how likely is it to do so, how bad might the consequences be, what should be done to maximize expected utility and minimize expected loss or regret, and how large are the remaining risks? In probabilistic causal models capable of representing unpredictable and novel events, probabilities for what will happen, and even what is possible, cannot necessarily be determined in advance. Standard decision and risk analysis questions become inherently unanswerable ("undecidable") for realistically complex causal systems with "open-world" uncertainties about what exists, what can happen, what other agents know, and how they will act. Recent artificial intelligence (AI) techniques enable agents (e.g., robots, drone swarms, and automatic controllers) to learn, plan, and act effectively despite open-world uncertainties in a host of practical applications, from robotics and autonomous vehicles to industrial engineering, transportation and logistics automation, and industrial process control. This article offers an AI/machine learning perspective on recent ideas for making decision and risk analysis (even) more useful. It reviews undecidability results and recent principles and methods for enabling intelligent agents to learn what works and how to complete useful tasks, adjust plans as needed, and achieve multiple goals safely and reasonably efficiently when possible, despite open-world uncertainties and unpredictable events. In the near future, these principles could contribute to the formulation and effective implementation of more effective plans and policies in business, regulation, and public policy, as well as in engineering, disaster management, and military and civil defense operations. They can extend traditional decision and risk analysis to deal more successfully with open-world novelty and unpredictable events in large-scale real-world planning, policymaking, and risk management.

show abstract

“…The second work [12] proposed an approach to learn behavior models as behavior trees for autonomous agents. The main goal of the proposal is to facilitate behavior modeling for autonomous agents in simulation and computer games.…”

Section: Mas and Learningmentioning

confidence: 99%

Multi-Agent Systems

Julián

Botti

2019

Applied Sciences

View full text Add to dashboard Cite

With the current advance of technology, agent-based applications are becoming a standard in a great variety of domains such as e-commerce, logistics, supply chain management, telecommunications, healthcare, and manufacturing. Another reason for the widespread interest in multi-agent systems is that these systems are seen as a technology and a tool that helps in the analysis and development of new models and theories in large-scale distributed systems or in human-centered systems. This last aspect is currently of great interest due to the need for democratization in the use of technology that allows people without technical preparation to interact with the devices in a simple and coherent way. In this Special Issue, different interesting approaches that advance this research discipline have been selected and presented.

show abstract

Learning Behavior Trees for Autonomous Agents with Hybrid Constraints Evolution

Abstract: The proposed approach can learn transparent behavior models represented as Behavior Trees, which could be used to alleviate the heaven endeavor of manual agent programming in game and simulation.

Cited by 23 publications

References 26 publications

Deliberative acting, planning and learning with hierarchical operational models

Deliberative acting, planning and learning with hierarchical operational models

Answerable and Unanswerable Questions in Risk Analysis with Open‐World Novelty

Multi-Agent Systems

Contact Info

Product

Resources

About