Neural networks for incremental dimensionality reduced reinforcement learning

Curran, William J.; Pocius, Rey; Smart, William D.

doi:10.1109/iros.2017.8205962

Cited by 7 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main function of SL is to generate a low-dimensional state space in which RL policy can perform well and be efficient. The studies [25][26][27][28][29] adapt SL methods to make the RL training process faster by separating the representation learning process and policy learning process.…”

Section: Related Workmentioning

confidence: 99%

Solving large-scale multi-agent tasks via transfer learning with dynamic state representation

Dou

Jia

Huang

2023

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

Many research results have emerged in the past decade regarding multi-agent reinforcement learning. These include the successful application of asynchronous advantage actor-critic, double deep Q-network and other algorithms in multi-agent environments, and the more representative multi-agent training method based on the classical centralized training distributed execution algorithm QMIX. However, in a large-scale multi-agent environment, training becomes a major challenge due to the exponential growth of the state-action space. In this article, we design a training scheme from small-scale multi-agent training to large-scale multi-agent training. We use the transfer learning method to enable the training of large-scale agents to use the knowledge accumulated by training small-scale agents. We achieve policy transfer between tasks with different numbers of agents by designing a new dynamic state representation network, which uses a self-attention mechanism to capture and represent the local observations of agents. The dynamic state representation network makes it possible to expand the policy model from a few agents (4 agents, 10 agents) task to large-scale agents (16 agents, 50 agents) task. Furthermore, we conducted experiments in the famous real-time strategy game Starcraft II and the multi-agent research platform MAgent. And also set unmanned aerial vehicles trajectory planning simulations. Experimental results show that our approach not only reduces the time consumption of a large number of agent training tasks but also improves the final training performance.

show abstract

Section: Related Workmentioning

confidence: 99%

Solving large-scale multi-agent tasks via transfer learning with dynamic state representation

Dou

Jia

Huang

2023

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

show abstract

“…The purpose of learning a latent space embedding is to reduce the dimensionality of the input state-space the policy network learns from. As the policy network gathers experience by interacting with the environment in real time, a reduction in the dimensionality of the state space leads to accelerated convergence [18].…”

Section: Learning Latent Space Representationmentioning

confidence: 99%

Reinforcement Learning for Shared Autonomy Drone Landings

Backman¹,

Kulić²,

Chung³

2022

Preprint

View full text Add to dashboard Cite

Novice pilots find it difficult to operate and land unmanned aerial vehicles (UAVs), due to the complex UAV dynamics, challenges in depth perception, lack of expertise with the control interface and additional disturbances from the ground effect. Therefore we propose a shared autonomy approach to assist pilots in safely landing a UAV under conditions where depth perception is difficult and safe landing zones are limited. Our approach comprises of two modules: a perception module that encodes information onto a compressed latent representation using two RGB-D cameras and a policy module that is trained with the reinforcement learning algorithm TD3 to discern the pilot's intent and to provide control inputs that augment the user's input to safely land the UAV. The policy module is trained in simulation using a population of simulated users. Simulated users are sampled from a parametric model with four parameters, which model a pilot's tendency to conform to the assistant, proficiency, aggressiveness and speed. We conduct a user study (n = 28) where human participants were tasked with landing a physical UAV on one of several platforms under challenging viewing conditions. The assistant, trained with only simulated user data, improved task success rate from 51.4% to 98.2% despite being unaware of the human participants' goal or the structure of the environment a priori. With the proposed assistant, regardless of prior piloting experience, participants performed with a proficiency greater than the most experienced unassisted participants.

show abstract

“…The purpose of latent space learning is to reduce the time required for the policy network to reach convergence. As the policy network gathers observations in real time, it is costly to explore large state spaces, therefore a reduction in the dimensionality of the state space leads to accelerated convergence [16]. Desired characteristics of a dimensionality reduction technique is that it is smooth, continuous and consistent [17], so that a policy network can efficiently learn how to perceive the encoded environment.…”

Section: Learning Latent Space Representation -Cross-modal Variationa...mentioning

confidence: 99%

Learning to Assist Drone Landings

Backman,

Kulić,

Chung

2020

Preprint

View full text Add to dashboard Cite

Unmanned aerial vehicles (UAVs) are often used for navigating dangerous terrains, however they are difficult to pilot. Due to complex input-output mapping schemes, limited perception, the complex system dynamics and the need to maintain a safe operation distance, novice pilots experience difficulties in performing safe landings in obstacle filled environments. Previous work has proposed autonomous landing methods however these approaches do not adapt to the pilot's control inputs and require the pilot's goal to be known a priori. In this work we propose a shared autonomy approach that assists novice pilots to perform safe landings on one of several elevated platforms at a proficiency equal to or greater than experienced pilots. Our approach consists of two modules, a perceptual module and a policy module. The perceptual module compresses high dimensionality RGB-D images into a latent vector trained with a cross-modal variational auto-encoder. The policy module provides assistive control inputs trained with the reinforcement algorithm TD3. We conduct a user study (n=33) where participants land a simulated drone on a specified platform out of five candidate platforms with and without the use of the assistant. Despite the goal platform not being known to the assistant, participants of all skill levels were able to outperform experienced participants while assisted in the task.

show abstract

Neural networks for incremental dimensionality reduced reinforcement learning

Cited by 7 publications

References 24 publications

Solving large-scale multi-agent tasks via transfer learning with dynamic state representation

Solving large-scale multi-agent tasks via transfer learning with dynamic state representation

Reinforcement Learning for Shared Autonomy Drone Landings

Learning to Assist Drone Landings

Contact Info

Product

Resources

About