Optimal Actor-Critic Policy With Optimized Training Datasets

Banerjee, Chayan; Chen, Zhiyong; Noman, Nasimul; Zamani, Mohsen

doi:10.1109/tetci.2022.3140375

Cited by 7 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, we penalise actions that lead to the structures that have energies that we have seen already, which is to promote general exploration of the potential energy surface. Although the last penalty is more general than the one before, it is practically useful to control these penalties independently, e.g., by removing one completely, or by imposing different penalty levels for zero reward cases where the action does not alter the structure and non-unique energy structures where the action does not promote exploration [25].…”

Section: Reinforcement Learning In Basin-hopping Cspmentioning

confidence: 99%

Reinforcement Learning in Crystal Structure Prediction

Zamaraeva

Collins

Antypov

et al. 2023

Preprint

View full text Add to dashboard Cite

Crystal Structure Prediction (CSP) is a fundamental computational problem in materials science. Basin-hopping is a prominent CSP method that combines global Monte Carlo sampling to search over candidate trial structures with local energy minimisation of these candidates. The sampling uses a stochastic policy to randomly choose which action (such as a swap of atoms) will be used to transform the current structure into the next. Typically hand-tuned for a specific system before the run starts, such a policy is simply a fixed discrete probability distribution of possible actions, which does not depend on the current structure and does not adapt during a CSP run. We show that reinforcement learning (RL) can generate a dynamic policy that both depends on the current structure and improves on the fly during the CSP run. We demonstrate the efficacy of our approach on two CSP codes, FUSE and MC-EMMA. Specifically, we show that, when applied to the autonomous exploration of a phase field to identify the accessible crystal structures, RL can save up to 46% of the computation time.

show abstract

Section: Reinforcement Learning In Basin-hopping Cspmentioning

confidence: 99%

Reinforcement Learning in Crystal Structure Prediction

Zamaraeva

Collins

Antypov

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Through trial-and-error interactions with the environment, Reinforcement Learning (RL) offers a promising approach to solving decision-making and optimization problems. Over the past few years, RL has accomplished impressive feats in handling difficult tasks, in such domains as autonomous driving [119,16], locomotion control [99,129], robotics [71,94], continuous control [5,6,7], and multi-agent systems and control [39,15]. A majority of these successful approaches are purely data-driven and leverage trial-and-error to freely explore the search space.…”

Section: Introductionmentioning

confidence: 99%

A Survey on Physics Informed Reinforcement Learning: Review and Open Problems

Banerjee,

Nguyen,

Fookes

et al. 2023

Preprint

View full text Add to dashboard Cite

“…Off-policy algorithms necessitate a substantial number of samples in the ER buffer to facilitate meaningful policy learning. Researchers have explored various exploration strategies to enhance sampling efficiency, such as action space perturbation employed in DDPG and TD3, as well as policy parameter perturbation in [10], [11]. While these strategies offer clear advantages, their effectiveness tends to be less consistent across diverse environmental settings, particularly in high-dimensional environments.…”

Section: Introductionmentioning

confidence: 99%

Enhancing Exploration in Actor-Critic Algorithms:An Approach to Incentivize Plausible Novel States

Banerjee,

Chen,

Noman

2023

Preprint

View full text Add to dashboard Cite

<p>Actor-critic (AC) algorithms are model-free deep reinforcement learning techniques that have consistently demon- strated their effectiveness across various domains, especially in addressing continuous control challenges. Enhancing exploration (action entropy) and exploitation (expected return) through more efficient sample utilization is pivotal in AC algorithms. The fundamental strategy of a learning algorithm is to intelligently navigate the environment’s state space, prioritizing the explo- ration of rarely visited states over frequently encountered ones. In alignment with this strategy, we propose an innovative approach to bolster exploration by employing an intrinsic reward based on a state’s novelty and the potential benefits of exploring that state, which we term plausible novelty. Our approach is designed for seamless integration into off-policy AC algorithms. Through incentivized exploration of plausibly novel states, an AC algo- rithm can substantially enhance its sample efficiency and overall training performance. The new approach is verified through extensive simulations across various continuous control tasks within MuJoCo environments, utilizing a range of prominent off-policy AC algorithmsIncentivizing Plausible Novel States: An Exploration Boosting Approach for Actor Critic Algorithms</p>

show abstract

Optimal Actor-Critic Policy With Optimized Training Datasets

Cited by 7 publications

References 18 publications

Reinforcement Learning in Crystal Structure Prediction

Reinforcement Learning in Crystal Structure Prediction

A Survey on Physics Informed Reinforcement Learning: Review and Open Problems

Enhancing Exploration in Actor-Critic Algorithms:An Approach to Incentivize Plausible Novel States

Contact Info

Product

Resources

About