SofaGym: An Open Platform for Reinforcement Learning Based on Soft Robot Simulations

Ménager, Etienne; Schegg, Pierre; Elie, Khairallah,; Marchal, Damien; Dequidt, Jérémie; Preux, Philippe; Duriez, Christian

doi:10.1089/soro.2021.0123

Cited by 22 publications

(11 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[ 39 ] A recenly developed platfrom using reinforcement learning in FEM have begun to challenge this. But to date, it has only been demonstrated in learning controllers, rather than designs [ 40 ]…”

Section: Related Workmentioning

confidence: 99%

Diversity‐Based Topology Optimization of Soft Robotic Grippers

Pinskier,

Wang,

Liow

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Soft grippers are ideal for grasping delicate, deformable objects with complex geometries. Universal soft grippers have proven effective for grasping common objects, however complex objects or environments require bespoke gripper designs. Multi‐material printing presents a vast design‐space which, when coupled with an expressive computational design algorithm, can produce numerous, novel, high‐performance soft grippers. Finding high‐performing designs in challenging design spaces requires tools that combine rapid iteration, simulation accuracy, and fine‐grained optimization across a range of gripper designs to maximize performance, no current tools meet all these criteria. Herein, a diversity‐based soft gripper design framework combining generative design and topology optimization (TO) are presented. Compositional pattern‐producing networks (CPPNs) seed a diverse set of initial material distributions for the fine‐grained TO. Focusing on vacuum‐driven multi‐material soft grippers, several grasping modes (e.g. pinching, scooping) emerging without explicit prompting are demonstrated. Extensive automated experimentation with printed multi‐material grippers confirms optimized candidates exceed the grasp strength of comparable commercial designs. Grip strength, durability, and robustness is evaluated across 15,170 grasps. The combination of fine‐grained generative design, diversity‐based design processes, high‐fidelity simulation, and automated experimental evaluation represents a new paradigm for bespoke soft gripper design which is generalizable across numerous design domains, tasks, and environments.

show abstract

Section: Related Workmentioning

confidence: 99%

Diversity‐Based Topology Optimization of Soft Robotic Grippers

Pinskier,

Wang,

Liow

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…Two initial attempts are conducted by injecting random noise in the state observations [10], and more recently to both observations and policy actions [12]. However, although encouraged by the community [11], [17], no randomization of dynamics parameters for parametric physics engines currently exists. The same statement applies to the investigation of ADR methods for parameters inference of deformable bodies.…”

Section: B Sim-to-real Transfer With Domain Randomizationmentioning

confidence: 99%

“…We test our method by building on recently introduced benchmark tasks in the domain of soft robotics for manipulation and locomotion [17]. In particular, we consider four evaluation domains for the purpose of our analysis: TrunkReach, TrunkPush, TrunkLift, MultiGait.…”

Section: A Tasksmentioning

confidence: 99%

“…However, the performance of learned arXiv:2303.04136v1 [cs.RO] 7 Mar 2023 policies in the real-world often falls short of expectations due to the differences between the simulation and the real-world, namely policies are affected by the reality gap. This is further exacerbated by the complexity of measuring deformable object parameters and designing accurate modeling, limiting current applications in soft robotics to overly simplified models for locomotion environments [17], [11] or simple trajectory tracking tasks [12]. Therefore, we identify the need to design novel sim-to-real transfer methods to scale the training of soft robots to more complex tasks, such as manipulation and contact-rich settings, and to overcome modeling approximations.…”

Section: Introductionmentioning

confidence: 99%

“…In this work, we present a thorough investigation of Domain Randomization in the context of sim-to-real transfer for closed-loop soft robot control. We include the examination of existing DR techniques for learning robust control policies on recently proposed soft robot benchmark environments [17], [22]-namely a reaching task for a trunk-like robot and walking for a MultiGait soft design. In this context, we design a novel extension to state-of-the-art ADR methods and evaluate its capabilities to infer complex deformable object parameters.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Online vs. Offline Adaptive Domain Randomization Benchmark

Tiboni

Arndt²,

Averta³

et al. 2023

Human-Friendly Robotics 2022

View full text Add to dashboard Cite

Soft robots are becoming extremely popular thanks to their intrinsic safety to contacts and adaptability. However, the potentially infinite number of Degrees of Freedom makes their modeling a daunting task, and in many cases only an approximated description is available. This challenge makes reinforcement learning (RL) based approaches inefficient when deployed on a realistic scenario, due to the large domain gap between models and the real platform. In this work, we demonstrate, for the first time, how Domain Randomization (DR) can solve this problem by enhancing RL policies with: i) a higher robustness w.r.t. environmental changes; ii) a higher affordability of learned policies when the target model differs significantly from the training model; iii) a higher effectiveness of the policy, which can even autonomously learn to exploit the environment to increase the robot capabilities (environmental constraints exploitation). Moreover, we introduce a novel algorithmic extension of previous adaptive domain randomization methods for the automatic inference of dynamics parameters for deformable objects. We provide results on four different tasks and two soft robot designs, opening interesting perspectives for future research on Reinforcement Learning for closed-loop soft robot control.

show abstract

Learning Controllers for Continuum Soft Manipulators: Impact of Modeling and Looming Challenges

Falotico,

Donato,

Alessi

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Soft manipulators, renowned for their compliance and adaptability, hold great promise in their ability to engage safely and effectively with intricate environments and delicate objects. Nonetheless, controlling these soft systems presents distinctive hurdles owing to their nonlinear behavior and complicated dynamics. Learning‐based controllers for continuum soft manipulators offer a viable alternative to model‐based approaches that may struggle to account for uncertainties and variability in soft materials, limiting their effectiveness in real‐world scenarios. Learning‐based controllers can be trained through experience, exploiting various forward models that differ in physical assumptions, accuracy, and computational cost. In this article, the key features of popular forward models, including geometrical, pseudo‐rigid, continuum mechanical, or learned, are first summarized. Then, a unique characterization of learning‐based policies, emphasizing the impact of forward models on the control problem and how the state of the art evolves, is offered. This leads to the presented perspectives outlining current challenges and future research trends for machine‐learning applications within soft robotics.

show abstract

SofaGym: An Open Platform for Reinforcement Learning Based on Soft Robot Simulations

Cited by 22 publications

References 60 publications

Diversity‐Based Topology Optimization of Soft Robotic Grippers

Diversity‐Based Topology Optimization of Soft Robotic Grippers

Online vs. Offline Adaptive Domain Randomization Benchmark

Learning Controllers for Continuum Soft Manipulators: Impact of Modeling and Looming Challenges

Contact Info

Product

Resources

About