Deep Reinforcement Learning: A State-of-the-Art Walkthrough

Abstract

Deep Reinforcement Learning is a topic that has gained a lot of attention recently, due to the unprecedented achievements and remarkable performance of such algorithms in various benchmark tests and environmental setups. The power of such methods comes from the combination of an already established and strong field of Deep Learning, with the unique nature of Reinforcement Learning methods. It is, however, deemed necessary to provide a compact, accurate and comparable view of these methods and their results for the means of gaining valuable technical and practical insights. In this work we gather the essential methods related to Deep Reinforcement Learning, extracting common property structures for three complementary core categories: a) Model-Free, b) Model-Based and c) Modular algorithms. For each category, we present, analyze and compare state-of-the-art Deep Reinforcement Learning algorithms that achieve high performance in various environments and tackle challenging problems in complex and demanding tasks. In order to give a compact and practical overview of their differences, we present comprehensive comparison figures and tables, produced by reported performances of the algorithms under two popular simulation platforms: the Atari Learning Environment and the MuJoCo physics simulation platform. We discuss the key differences of the various kinds of algorithms, indicate their potential and limitations, as well as provide insights to researchers regarding future directions of the field.

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

“…We next present the basic components of certain important methods that are commonly used in the field of wireless communications and RIS-empowered communication systems. A detailed walkthrough of the state-of-the-art in DRL approaches can be found in the recent survey [161].…”

Exploration in Deep Reinforcement Learning: A Survey

“…Several review articles exist in the field of reinforcement learning. Aubert et al [16] presented an overview of intrinsic motivation in reinforcement learning, Li [17] presented a comprehensive overview of techniques and applications, Nguyen et al [18] considered an application to multi-agent problems, Levine [19] provided a tutorial and extensive comparison with probabilistic inference methods and [20] provided an extensive description of the key breakthrough methods in reinforcement learning, including ones in exploration. However, none of the aforementioned reviews focused on exploration or considered it in great detail.…”

REIN-2: Giving Birth to Prepared Reinforcement Learning Agents Using Reinforcement Learning Agents

“…In the recent years, Deep Learning methods were incorporated into traditional RL techniques to create the field of Deep Reinforcement Learning (Deep RL). Such methodologies lead to extraordinary results in solving highly complex problems and indicated new, promising directions towards building powerful Artificial General Intelligence systems [Lazaridis et al, 2020].…”

“…One such critical problem is the lack of sample efficiency during the training procedure, since even the most sample-efficient state-of-the-art Deep RL models require large amounts of interactions with an environment (i.e. experience) until satisfying performance is achieved, especially when the environment dynamics are highly complex [Lazaridis et al, 2020]. This constitutes a major drawback in cases where the cost of performing wrong actions is high, and thus effective training is essentially impossible.…”