Exploiting dream-like simulation mechanisms to develop safer agents for automated driving: The “Dreams4Cars” EU research and innovation action

Lio, Mauro Da; Mazzalai, Alessandro; Windridge, David; Thill, Serge; Svensson, Henrik; Yüksel, Mehmed; Gurney, Kevin; Saroldi, Andrea; Andreone, Luisa; Anderson, Sean; Heich, Hermann-Josef

doi:10.1109/itsc.2017.8317649

Cited by 8 publications

(5 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Self-driving cars have been an ideal test case for many practical applications of machine learning and cognitive systems in recent times, not least because they promise an autonomous agent solving non-trivial problems in a real environment (Mahmoud et al, 2022 ). However, many interesting challenges remain unsolved, including how vehicles can learn autonomously and robustly to drive safely even in rare events or situations that the system developer does not anticipate (Da Lio et al, 2017 ).…”

Section: Discussionmentioning

confidence: 99%

“…As with all such real-world situations, it is not feasible (nor desirable to attempt) to fully and completely anticipate all possible situations that an autonomous vehicle might possibly encounter. Therefore, fully hard-coded solutions are simply impractical (Da Lio et al, 2017 ) and a significant portion of the research on autonomous vehicle control focuses on machine learning approaches, such as deep learning, to design a controller that can learn the appropriate skills. Deep learning, generally speaking, has been hugely successful in many tasks that are relevant for autonomous vehicles, most famously image processing (Geiger et al, 2013 ; Wali et al, 2015 ; Grigorescu et al, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

“…There is thus a significant interest in training controllers for autonomous vehicles in simulation with auto-generated scenarios. Such scenarios can be entirely new situations or variations of actually encountered rare events to increase robustness (Da Lio et al, 2017 ). For example, initial training in tasks like lane-keeping could involve entirely new situations (if the vehicle encounters certain curvatures for the first time).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers

Mahmoud

Billing

Svensson

et al. 2023

Front. Artif. Intell.

View full text Add to dashboard Cite

Learning from only real-world collected data can be unrealistic and time consuming in many scenario. One alternative is to use synthetic data as learning environments to learn rare situations and replay buffers to speed up the learning. In this work, we examine the hypothesis of how the creation of the environment affects the training of reinforcement learning agent through auto-generated environment mechanisms. We take the autonomous vehicle as an application. We compare the effect of two approaches to generate training data for artificial cognitive agents. We consider the added value of curriculum learning—just as in human learning—as a way to structure novel training data that the agent has not seen before as well as that of using a replay buffer to train further on data the agent has seen before. In other words, the focus of this paper is on characteristics of the training data rather than on learning algorithms. We therefore use two tasks that are commonly trained early on in autonomous vehicle research: lane keeping and pedestrian avoidance. Our main results show that curriculum learning indeed offers an additional benefit over a vanilla reinforcement learning approach (using Deep-Q Learning), but the replay buffer actually has a detrimental effect in most (but not all) combinations of data generation approaches we considered here. The benefit of curriculum learning does depend on the existence of a well-defined difficulty metric with which various training scenarios can be ordered. In the lane-keeping task, we can define it as a function of the curvature of the road, in which the steeper and more occurring curves on the road, the more difficult it gets. Defining such a difficulty metric in other scenarios is not always trivial. In general, the results of this paper emphasize both the importance of considering data characterization, such as curriculum learning, and the importance of defining an appropriate metric for the task.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers

Mahmoud

Billing

Svensson

et al. 2023

Front. Artif. Intell.

View full text Add to dashboard Cite

show abstract

“…The neural network approximant will learn mapping the lane geometry to the activation pattern (salience). One early example of this was given in [35]. Another example of training neural network approximants may also be found in [36].…”

Section: ) Computation Of the "Motor Cortex": Declarative Predictionsmentioning

confidence: 99%

Agent Architecture for Adaptive Behaviors in Autonomous Driving

et al. 2020

Self Cite

View full text Add to dashboard Cite

Evolution has endowed animals with outstanding adaptive behaviours which are grounded in the organization of their sensorimotor system. This paper uses inspiration from these principles of organization in the design of an artificial agent for autonomous driving. After distilling the relevant principles from biology, their functional role in the implementation of an artificial system are explained. The resulting Agent, developed in an EU H2020 Research and Innovation Action, is used to concretely demonstrate the emergence of adaptive behaviour with a significant level of autonomy. Guidelines to adapt the same principled organization of the sensorimotor system to other agents for driving are also obtained. The demonstration of the system abilities is given with example scenarios and open access simulation tools. Prospective developments concerning learning via mental imagery are finally discussed.

show abstract

“…In [7], the authors propose a simulation tool of cooperative maneuvers among autonomous vehicles in which virtual and real vehicles can conjunctively interact. Another work found in literature use real data collected from autonomous vehicles for creating new simulation scenarios and improve driving behaviors [10].…”

Section: Introductionmentioning

confidence: 99%

Framework for motion prediction of vehicles in a simulation environment

Lee¹,

Trentin²,

Villagrá³

2020

XL Jornadas De Automática: Libro De Actas (Ferrol, 4-6 De Septiembre De 2019)

View full text Add to dashboard Cite

Efficient testing and validation of software components for highly automate vehicles is one of the key challenges to be solved for their massive deployment. The number of driving situation and environment variables makes validation almost intractable with real vehicles in open roads, and the testing reproducibility can only be achieved via simulation. This manuscript presents a framework and preliminary results for motion prediction of vehicles in a simulation environment that is being currently developed by the AUTOPIA Program.

show abstract

Exploiting dream-like simulation mechanisms to develop safer agents for automated driving: The “Dreams4Cars” EU research and innovation action

Cited by 8 publications

References 43 publications

How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers

How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers

Agent Architecture for Adaptive Behaviors in Autonomous Driving

Framework for motion prediction of vehicles in a simulation environment

Contact Info

Product

Resources

About