On the utility of dreaming: A general model for how learning in artificial agents can benefit from data hallucination

Windridge, David; Svensson, Henrik; Thill, Serge

doi:10.1177/1059712319896489

Cited by 5 publications

(5 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In evaluation, the agent performed better when trained in the synthetic (unseen) environment before being injected back into the real environment. This illustrates that agents might benefit from unseen examples, similar to how humans gain a behavioral advantage from re-enacting hypothetical future situations (Revonsuo, 2000 ; Svensson et al, 2013 ; Billing et al, 2016 ; Gershman and Daw, 2017 ; Windridge et al, 2020 ).…”

Section: Introductionmentioning

confidence: 83%

How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers

Mahmoud

Billing

Svensson

et al. 2023

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

Learning from only real-world collected data can be unrealistic and time consuming in many scenario. One alternative is to use synthetic data as learning environments to learn rare situations and replay buffers to speed up the learning. In this work, we examine the hypothesis of how the creation of the environment affects the training of reinforcement learning agent through auto-generated environment mechanisms. We take the autonomous vehicle as an application. We compare the effect of two approaches to generate training data for artificial cognitive agents. We consider the added value of curriculum learning—just as in human learning—as a way to structure novel training data that the agent has not seen before as well as that of using a replay buffer to train further on data the agent has seen before. In other words, the focus of this paper is on characteristics of the training data rather than on learning algorithms. We therefore use two tasks that are commonly trained early on in autonomous vehicle research: lane keeping and pedestrian avoidance. Our main results show that curriculum learning indeed offers an additional benefit over a vanilla reinforcement learning approach (using Deep-Q Learning), but the replay buffer actually has a detrimental effect in most (but not all) combinations of data generation approaches we considered here. The benefit of curriculum learning does depend on the existence of a well-defined difficulty metric with which various training scenarios can be ordered. In the lane-keeping task, we can define it as a function of the curvature of the road, in which the steeper and more occurring curves on the road, the more difficult it gets. Defining such a difficulty metric in other scenarios is not always trivial. In general, the results of this paper emphasize both the importance of considering data characterization, such as curriculum learning, and the importance of defining an appropriate metric for the task.

show abstract

Section: Introductionmentioning

confidence: 83%

How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers

Mahmoud

Billing

Svensson

et al. 2023

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…DD is most widely used for manipulating images, whether it is used for creating hallucination images [8], [23]- [25], or art images [26]. Many researchers employed the DD algorithm with deep learning techniques to produce images where the DD algorithm works to transform the strange landforms in one image (i.e., conical sandstone) into different shapes (i.e., faces, animals, and so on) [5].…”

Section: B Deep Dream For Imagementioning

confidence: 99%

A Systematic Review of Deep Dream

2023

IJCCCE

View full text Add to dashboard Cite

Deep Dream (DD) is a new technology that works as a creative image-editing approach by employing the representations of CNN to produce dreams-like images by taking the benefits of both Deep CNN and Inception to build the dream through layer-by-layer implementation. As the days go by, the DD becomes widely used in the artificial intelligence (AI) fields. This paper is the first systematic review of DD. We focused on the definition, importance, background, and applications of DD. Natural language processing (NLP), images, videos, and audio are the main fields in which DD is applied. We also discussed the main concepts of the DD, like transfer learning and Inception. We addressed the contributions, databases, and techniques that have been used to build the models, the limitations, and evaluation metrics for each one of the included research papers. Finally, some interesting recommendations have been listed to serve the researchers in the future. Index Terms— Deep dream, deep CNN, gradient ascent, Inception, style transfer.

show abstract

“…They include the ability to imagine -at a deferred time-hypothetical situations to develop action strategies for them in advance. This ability allows acting correctly in conditions not met before without needing time to carry out mental simulations at that moment, which might be a time-critical context [15], [28].…”

Section: B Learning Via Mental Simulationmentioning

confidence: 99%

“…For a more in-depth discussion on the utility of offline mental simulations, see [28]; for a discussion of different learning mechanisms at sleep/dream, see also [30].…”

Section: B Learning Via Mental Simulationmentioning

confidence: 99%

A Mental Simulation Approach for Learning Neural-Network Predictive Control (in Self-Driving Cars)

et al. 2020

Self Cite

View full text Add to dashboard Cite

This paper presents a novel approach to learning predictive motor control via "mental simulations". The method, inspired by learning via mental imagery in natural Cognition, develops in two phases: first, the learning of predictive models based on data recorded in the interaction with the environment; then, at a deferred time, the synthesis of inverse models via offline episodic simulations. Parallelism with human-engineered control-theoretic workflow (mathematical modeling the direct dynamics followed by optimal control inversion) is established. Compared to the latter human-directed synthesis, the mental simulation approach increases autonomy: a robotic agent can learn predictive models and synthesize inverse ones with a large degree of independence. Human modeling is still needed but limited to providing efficient templates for the forward and inverse neural networks and a few other directives. One could consider these templates as the efficient brain network typologies that evolution produced to permit live beings quickly and efficiently learning. The structure of the neural networks-both forward and inverse ones-is made of interpretable "local models", which follows the cerebellar organization (and are also similar to local model approaches known in the literature). We demonstrate the learning of a first-round model (contrasted to Model Predictive Control) for lateral vehicle dynamics. Then, we demonstrate a second learning iteration, where the forward/inverse neural models are significantly improved.

show abstract

On the utility of dreaming: A general model for how learning in artificial agents can benefit from data hallucination

Cited by 5 publications

References 26 publications

How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers

How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers

A Systematic Review of Deep Dream

A Mental Simulation Approach for Learning Neural-Network Predictive Control (in Self-Driving Cars)

Contact Info

Product

Resources

About