Distributed Learning for Planning Under Uncertainty Problems with Heterogeneous Teams

Abstract

This paper considers the problem of multiagent sequential decision making under uncertainty and incomplete knowledge of the state transition model. A distributed learning framework, where each agent learns an individual model and shares the results with the team, is proposed. The challenges associated with this approach include choosing the model representation for each agent and how to effectively share these representations under limited communication. A decentralized extension of the model learning scheme based on the Incremental Feature Dependency Discovery (Dec-iFDD) is presented to address the distributed learning problem. The representation selection problem is solved by leveraging iFDD's property of adjusting the model complexity based on the observed data. The model sharing problem is addressed by having each agent rank the features of their representation based on the model reduction error and broadcast the most relevant features to their teammates. The algorithm is tested on the multiagent block building and the persistent search and track missions. The results show that the proposed distributed learning scheme is particularly useful in heterogeneous learning setting, where each agent learns significantly different models. We show through large-scale planning under uncertainty simulations and flight experiments with state-dependent actuator and fuel-burn-rate uncertainty that our planning approach can outperform planners that do not account for heterogeneity between agents.

Contextual Awareness in Human-Advanced-Vehicle Systems: A Survey

“…Bacciu et al [60] proposed a recurrent neural network approach to planning for in-home assistance robots. In Ure et al [61] and Tont [62] the problem of navigation planning under uncertainty in multi-agent systems is addressed. In [61] decentralised learning algorithms aggregate their knowledge by sharing weightings with a centralized planning algorithm.…”

“…In Ure et al [61] and Tont [62] the problem of navigation planning under uncertainty in multi-agent systems is addressed. In [61] decentralised learning algorithms aggregate their knowledge by sharing weightings with a centralized planning algorithm. In [62] Bayesian belief networks and probabilistic inference are used to model complex relationships between mobile agents.…”

Centralized model predictive control with distributed adaptation

“…To the best of our knowledge a framework in which a discrete adaptive controller interacts with MPC only at discrete instants of time is missing in literature. The planning-learning framework presented in [17] for cooperative multi-agent systems is also relevant to the context of the present article. In [17], each agent learns different models and shares some relevant information with the team.…”

“…The planning-learning framework presented in [17] for cooperative multi-agent systems is also relevant to the context of the present article. In [17], each agent learns different models and shares some relevant information with the team. Differently from [17], here we consider noninteracting agents with known dynamics, MPC is employed for planning, and adaptive control for learning only the disturbance weights.…”

“…In [17], each agent learns different models and shares some relevant information with the team. Differently from [17], here we consider noninteracting agents with known dynamics, MPC is employed for planning, and adaptive control for learning only the disturbance weights.…”