FORK: A FORward-looKing Actor for Model-Free Reinforcement Learning

Wei, Honghao; Lü, Ying

doi:10.1109/cdc45484.2021.9683288

Cited by 3 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Agents are trained with three different seeds for 3K episodes, and the one with the best average reward is selected as the baseline. This agent reaches an average over the last 100 episodes above 300+ in about 0.5M steps similar to the results reported in Wei & Ying (2021). All the network weights are transferred to the retrained agent.…”

Section: Lunarlandercontinuoussupporting

confidence: 63%

“…We use SAC-I to learn as well as to weight that negative reward. The BipedalWalkerHardcore-v3 is a challenging task, known to be unsolvable for many of the simpler non-recurrent DRL architectures or model-free RL methods (Wei & Ying, 2021). We solve the task using a standard two-layer dense-network architecture, and adopt two strategies: removing the fall-penalty and creating a cumulative version of the task reward.…”

Section: Bipedalwalkerhardcore-v3mentioning

confidence: 99%

See 1 more Smart Citation

Soft Actor-Critic with Inhibitory Networks for Faster Retraining

Ide¹,

Mićović²,

Guarino³

et al. 2022

Preprint

View full text Add to dashboard Cite

Reusing previously trained models is critical in deep reinforcement learning to speed up training of new agents. However, it is unclear how to acquire new skills when objectives and constraints are in conflict with previously learned skills. Moreover, when retraining, there is an intrinsic conflict between exploiting what has already been learned and exploring new skills. In soft actor-critic (SAC) methods, a temperature parameter can be dynamically adjusted to weight the action entropy and balance the explore × exploit trade-off. However, controlling a single coefficient can be challenging within the context of retraining, even more so when goals are contradictory. In this work, inspired by neuroscience research, we propose a novel approach using inhibitory networks to allow separate and adaptive state value evaluations, as well as distinct automatic entropy tuning. Ultimately, our approach allows for controlling inhibition to handle conflict between exploiting less risky, acquired behaviors and exploring novel ones to overcome more challenging tasks. We validate our method through experiments in OpenAI Gym environments.

show abstract

Section: Lunarlandercontinuoussupporting

confidence: 63%

Section: Bipedalwalkerhardcore-v3mentioning

confidence: 99%

Soft Actor-Critic with Inhibitory Networks for Faster Retraining

Ide¹,

Mićović²,

Guarino³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…AI‐EDGE researchers have developed several innovative algorithms to handle such constraints while maintaining the best possible performance. For example, Wei, Liu, and Ying (2022) developed model‐free RL algorithms with optimal regret and constraint violation guarantees. Ghosh, Zhou, and Shroff (2022) further expanded our exploration to larger, potentially infinite, state spaces via a new model‐free constrained RL algorithm that also enjoys near‐optimal performance bounds.…”

Section: Research Themes Of Ai For Networkmentioning

confidence: 99%

AI‐EDGE: An NSF AI institute for future edge networks and distributed intelligence

Ju,

Li,

Liang

et al. 2024

AI Magazine

View full text Add to dashboard Cite

This paper highlights the overall endeavors of the NSF AI Institute for Future Edge Networks and Distributed Intelligence (AI‐EDGE) to create a research, education, knowledge transfer, and workforce development environment for developing technological leadership in next‐generation edge networks (6G and beyond) and artificial intelligence (AI). The research objectives of AI‐EDGE are twofold: “AI for Networks” and “Networks for AI.” The former develops new foundational AI techniques to revolutionize technologies for next‐generation edge networks, while the latter develops advanced networking techniques to enhance distributed and interconnected AI capabilities at edge devices. These research investigations are conducted across eight symbiotic thrust areas that work together to address the main challenges towards those goals. Such a synergistic approach ensures a virtuous research cycle so that advances in one area will accelerate advances in the other, thereby paving the way for a new generation of networks that are not only intelligent but also efficient, secure, self‐healing, and capable of solving large‐scale distributed AI challenges. This paper also outlines the institute's endeavors in education and workforce development, as well as broadening participation and enforcing collaboration.

show abstract

“…Concludingly, ( 5) is of precisely quadratic order in the monomials of the input vector. Our guiding principle is to reduce the number of weights compared to the three layer actor policy MLP network from [37] i.e. two hidden layers with hidden unit dimension 256 each.…”

Section: Q-mlp -Quadratic-multi-layer-perceptronmentioning

confidence: 99%

“…The solid curves in all figures shows the average cumulative rewards per episode, and the shaded region represents the standard deviations. The implementation of the TD3 and SAC algorithms as well as hyper parameter settings are taken from [37] 5 . The latter is closely based on [8] 6 in the case of the TD3 algorithm and [40] 7 for the SAC algorithm.…”

Section: A Training Environments and Rl Algorithmmentioning

confidence: 99%

A Quadratic Actor Network for Model-Free Reinforcement Learning

Weissenbacher¹,

Kawahara²

2021

Preprint

View full text Add to dashboard Cite

In this work we discuss the incorporation of quadratic neurons into policy networks in the context of modelfree actor-critic reinforcement learning. Quadratic neurons admit an explicit quadratic function approximation in contrast to conventional approaches where the the non-linearity is induced by the activation functions. We perform empiric experiments on several MuJoCo continuous control tasks and find that when quadratic neurons are added to MLP policy networks those outperform the baseline MLP whilst admitting a smaller number of parameters. The top returned reward is in average increased by 5.8% while being about 21% more sample efficient. Moreover, it can maintain its advantage against added action and observation noise.1 https://github.com/matthias-weissenbacher/ Quadratic_MLPs_in_RL 2 Experiments for a smaller number of environments.

show abstract

FORK: A FORward-looKing Actor for Model-Free Reinforcement Learning

Cited by 3 publications

References 7 publications

Soft Actor-Critic with Inhibitory Networks for Faster Retraining

Soft Actor-Critic with Inhibitory Networks for Faster Retraining

AI‐EDGE: An NSF AI institute for future edge networks and distributed intelligence

A Quadratic Actor Network for Model-Free Reinforcement Learning

Contact Info

Product

Resources

About