Crossing the reality gap in evolutionary robotics by promoting transferable controllers

Koos, Sylvain; Mouret, Jean-Baptiste; Doncieux, Stéphane

doi:10.1145/1830483.1830505

Cited by 91 publications

(81 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When the target is a real robotic platform, the inevitable discrepancies between the simulated robot and the real one introduce a new problem: controllers generated in simulations will be adapted to the simulation but not necessarily to the real robot. If they exploit a feature that is specific to the simulation, the behavior on the real robot will be less effective or maybe completely ineffective, thus leading to the reality gap problem [89,97,98].…”

Section: Reality Gapmentioning

confidence: 99%

“…While still keeping a constant simulation, another approach consists of evaluating several solutions directly on the real robot [98,97,133,142]. Relying on the hypothesis that reasonably good simulators do indeed exist, the approach proposes learning a model of behavior discrepancies between simulation and reality in order to avoid the most unrealistic behaviors.…”

Section: Reality Gapmentioning

confidence: 99%

“…This predicted transferability is used as a new objective in a multi-objective EA alongside other objectives so that generated solutions tend to behave the same in simulation and in reality. The approach has been applied to a quadruped [98,97] and biped [142] locomotion tasks as well as to a T-maze navigation task [98]. As the number of evaluations on the real robot is reduced, this approach is considered to also address the reducing evaluation challenge.…”

Section: Reality Gapmentioning

confidence: 99%

See 2 more Smart Citations

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

Doncieux

Mouret

2014

Evol. Intel.

Self Cite

View full text Add to dashboard Cite

Evolutionary robotics is often viewed as the application of a family of black-box optimization algorithms -evolutionary algorithms -to the design of robots, or parts of robots. When considering evolutionary robotics as black-box optimization, the selective pressure is mainly driven by a user-defined, black-box fitness function, and a domain-independent selection procedure. However, most evolutionary robotics experiments face similar challenges in similar setups: the selective pressure, and, in particular, the fitness function, is not a pure user-defined black box. The present review shows that, because evolutionary robotics experiments share common features, selective pressures for evolutionary robotics are a subject of research on their own. The literature has been split into two categories: goal refiners, aimed at changing the definition of a good solution, and process helpers, designed to help the search process. Two subcategories are further considered: task-specific approaches, which require knowledge on how to solve the task and taskagnostic ones, which do not need it. Besides highlighting the diversity of the approaches and their respective goals, the present review shows that many task-agnostic process helpers have been proposed during the last years, thus bringing us closer to the goal of a fully automated robot behavior design process.

show abstract

Section: Reality Gapmentioning

confidence: 99%

Section: Reality Gapmentioning

confidence: 99%

Section: Reality Gapmentioning

confidence: 99%

See 1 more Smart Citation

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

Doncieux

Mouret

2014

Evol. Intel.

Self Cite

View full text Add to dashboard Cite

show abstract

“…A better alternative is to make such behaviors less likely to evolve by incorporating transfer experiments from the beginning of evolution, e.g. by utilizing a multi-objective evolutionary algorithm that optimizes both a task-dependent controller fitness as well as a measure of how well the controller transfers from simulation to reality [21]. In any given generation, this method chooses at most one controller based on behavioral diversity to be evaluated on the real robot, requiring only a small number of hardware evaluations.…”

Section: Evolving Controllers For Physical Robotsmentioning

confidence: 99%

Constructing controllers for physical multilegged robots using the ENSO neuroevolution approach

et al. 2012

View full text Add to dashboard Cite

Evolving controllers for multilegged robots in simulation is convenient and flexible, making it possible to prototype ideas rapidly. However, transferring the resulting controllers to physical robots is challenging because it is difficult to simulate realworld complexities with sufficient accuracy. This paper bridges this gap by utilizing the Evolution of Network Symmetry and mOdularity (ENSO) approach to evolve modular neural network controllers that are robust to discrepancies between simulation and reality. This approach was evaluated by building a physical quadruped robot and by evolving controllers for it in simulation. An approximate model of the robot and its environment was built in a physical simulation and uncertainties in the real world were modeled as noise. The resulting controllers produced well-synchronized trot gaits when they were transferred to the physical robot, even on different walking surfaces. In contrast to a hand-designed PID controller, the evolved controllers also generalized well to changes in experimental conditions such as loss of voltage and were more robust against faults such as loss of a leg, making them strong candidates for real-world applications.

show abstract

“…The success in simulating highly dynamic motion in computer animation, however, has not been transferred in full to robotics. The discrepancy between what can be achieved in simulation and that in real world is referred as the "Reality Gap" in the Evolutionary Robotics community [22,14]. Researchers have put forth a long list of possible factors that give rise to the Reality Gap, such as simplified dynamic models, inaccurate model parameters, approximated hardware limitations, the absence of uncertainty and latency in sensors and actuators, and other unmodelled factors.…”

Section: Introductionmentioning

confidence: 99%

Preparing for the Unknown: Learning a Universal Policy with Online System Identification

Yu¹,

Tan²,

Liu³

et al. 2017

Robotics: Science and Systems XIII

179

148

View full text Add to dashboard Cite

Abstract-We present a new method of learning control policies that successfully operate under unknown dynamic models. We create such policies by leveraging a large number of training examples that are generated using a physical simulator. Our system is made of two components: a Universal Policy (UP) and a function for Online System Identification (OSI). We describe our control policy as universal because it is trained over a wide array of dynamic models. These variations in the dynamic model may include differences in mass and inertia of the robots components, variable friction coefficients, or unknown mass of an object to be manipulated. By training the Universal Policy with this variation, the control policy is prepared for a wider array of possible conditions when executed in an unknown environment. The second part of our system uses the recent state and action history of the system to predict the dynamics model parameters µ. The value of µ from the Online System Identification is then provided as input to the control policy (along with the system state). Together, UP-OSI is a robust control policy that can be used across a wide range of dynamic models, and that is also responsive to sudden changes in the environment. We have evaluated the performance of this system on a variety of tasks, including the problem of cart-pole swing-up, the double inverted pendulum, locomotion of a hopper, and block-throwing of a manipulator. UP-OSI is effective at these tasks across a wide range of dynamic models. Moreover, when tested with dynamic models outside of the training range, UP-OSI outperforms the Universal Policy alone, even when UP is given the actual value of the model dynamics. In addition to the benefits of creating more robust controllers, UP-OSI also holds out promise of narrowing the Reality Gap between simulated and real physical systems.

show abstract

Crossing the reality gap in evolutionary robotics by promoting transferable controllers

Cited by 91 publications

References 19 publications

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

Constructing controllers for physical multilegged robots using the ENSO neuroevolution approach

Preparing for the Unknown: Learning a Universal Policy with Online System Identification

Contact Info

Product

Resources

About