Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillation

Chen, Letian; Paleja, Rohan; Ghuy, Muyleng; Gombolay, Matthew

doi:10.1145/3319502.3374791

Cited by 20 publications

(15 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent advances in learning from demonstrations has introduced methods to capture heterogeneity from expert demonstrations. Prior work has showcased the ability to infer discrete modes of operation [28], multiple visual intentions [29], and most commonly diverse user preferences [30]- [33] from heterogeneous demonstrations. Existing approaches that embrace heterogeneity in demonstrations focus on inferring and encoding the distinct characteristics.…”

Section: Heterogeneous Learning From Demonstrationsmentioning

confidence: 99%

Resource-Aware Adaptation of Heterogeneous Strategies for Coalition Formation

Anusha¹,

Ravichandar²

2021

Preprint

View full text Add to dashboard Cite

Existing approaches to coalition formation often assume that requirements associated with tasks are precisely specified by the human operator. However, prior work has demonstrated that humans, while extremely adept at solving complex problems, struggle to explicitly state their solution strategy. In this work, we propose a framework to learn implicit task requirements directly from expert demonstrations of coalition formation. We also account for the fact that demonstrators may utilize different, equally-valid solutions to the same task. Essentially, we contribute a framework to model and infer such heterogeneous strategies to coalition formation. Next, we develop a resource-aware approach to generalize the inferred strategies to new teams without requiring additional training. To this end, we formulate and solve a constrained optimization problem that simultaneously selects the most appropriate strategy for a given target team, and optimizes the constituents of its coalitions accordingly. We evaluate our approach against several baselines, including some that resemble existing approaches, using detailed numerical simulations, StarCraft II battles, and a multi-robot emergencyresponse scenario. Our results indicate that our framework consistently outperforms all baselines in terms of requirement satisfaction, resource utilization, and task success rates.

show abstract

Section: Heterogeneous Learning From Demonstrationsmentioning

confidence: 99%

Resource-Aware Adaptation of Heterogeneous Strategies for Coalition Formation

Anusha¹,

Ravichandar²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Unlike these works, we consider the problem of learning multiple rewards from a dataset containing several unlabeled behaviour intents [3]. We highlight that a related but distinct problem is that of Meta-IRL, where the goal is to learn a good meta-reward prior [14,39] or shared base reward [6] that can be used to learn new rewards efficiently.…”

Section: Related Workmentioning

confidence: 99%

LiMIIRL: Lightweight Multiple-Intent Inverse Reinforcement Learning

Snoswell

Singh²,

2021

Preprint

View full text Add to dashboard Cite

Multiple-Intent Inverse Reinforcement Learning (MI-IRL) seeks to find a reward function ensemble to rationalize demonstrations of different but unlabelled intents. Within the popular expectation maximization (EM) framework for learning probabilistic MI-IRL models, we present a warm-start strategy based on up-front clustering of the demonstrations in feature space. Our theoretical analysis shows that this warm-start solution produces a near-optimal reward ensemble, provided the behavior modes satisfy mild separation conditions. We also propose a MI-IRL performance metric that generalizes the popular Expected Value Difference measure to directly assesses learned rewards against the ground-truth reward ensemble. Our metric elegantly addresses the difficulty of pairing up learned and ground truth rewards via a min-cost flow formulation, and is efficiently computable. We also develop a MI-IRL benchmark problem that allows for more comprehensive algorithmic evaluations. On this problem, we find our MI-IRL warm-start strategy helps avoid poor quality local minima reward ensembles, resulting in a significant improvement in behavior clustering. Our extensive sensitivity analysis demonstrates that the quality of the learned reward ensembles is improved under various settings, including cases where our theoretical assumptions do not necessarily hold. Finally, we demonstrate the effectiveness of our methods by discovering distinct driving styles in a large real-world dataset of driver GPS trajectories.Preprint. Under review.

show abstract

“…Missions and games in FireCommander do not have unique solutions and each category of agents must learn both a local team-objective (e.g., either perform sensing or acting) as well as a global composite-objective (e.g., fight the fire and protect the facilities). Accordingly, the game objectives can be interpreted and interacted with in various different ways by a user and thus, FireCommander can be a perfect environment for developing learning from heterogeneous demonstrations algorithms [20,21,22,23,24], particularly for multi-agent coordination. 3.…”

Section: Stochastic and Probabilistic Environmentmentioning

confidence: 99%

“…• Multi-agent Learning from Heterogeneous Demonstrations (MA-LfHD)-FireCommander inherently includes two categories of heterogeneous robots: (1) Perception (e.g., Sensing) agents and ( 2) Action (e.g., Manipulator) agents, and is a multi-objective game. Missions and games in FireCommander do not have unique solutions and thus, can be interpreted and interacted with in various different ways by a user and thus, can be a perfect test-bed for developing learning from heterogeneous demonstrations algorithms [69,20,70,21,22,23,24,71].…”

Section: Multi-agent Learning From Demonstrations: Inverse Rlmentioning

confidence: 99%

FireCommander: An Interactive, Probabilistic Multi-agent Environment for Heterogeneous Robot Teams

Seraj¹,

Wu²,

Gombolay³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

The purpose of this tutorial is to help individuals use the FireCommander game environment for research applications. The FireCommander is an interactive, probabilistic joint perceptionaction reconnaissance environment in which a composite team of agents (e.g., robots) cooperate to fight dynamic, propagating firespots (e.g., targets). In FireCommander game, a team of agents must be tasked to optimally deal with a wildfire situation in an environment with propagating fire areas and some facilities such as houses, hospitals, power stations, etc. The team of agents can accomplish their mission by first sensing (e.g., estimating fire states), communicating the sensed fire-information among each other and then taking action to put the firespots out based on the sensed information (e.g., dropping water on estimated fire locations). The FireCommander environment can be useful for research topics spanning a wide range of applications from Reinforcement Learning (RL) and Learning from Demonstration (LfD), to Coordination, Psychology, Human-Robot Interaction (HRI) and Teaming. There are four important facets of the FireCommander environment that overall, create a non-trivial game:1.

show abstract

Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillation

Cited by 20 publications

References 17 publications

Resource-Aware Adaptation of Heterogeneous Strategies for Coalition Formation

Resource-Aware Adaptation of Heterogeneous Strategies for Coalition Formation

LiMIIRL: Lightweight Multiple-Intent Inverse Reinforcement Learning

FireCommander: An Interactive, Probabilistic Multi-agent Environment for Heterogeneous Robot Teams

Contact Info

Product

Resources

About