FishGym: A High-Performance Physics-based Simulation Framework for Underwater Robot Learning

Abstract

Bionic underwater robots have demonstrated their superiority in many applications. Yet, training their intelligence for a variety of tasks that mimic the behavior of underwater creatures poses a number of challenges in practice, mainly due to lack of a large amount of available training data as well as the high cost in real physical environment. Alternatively, simulation has been considered as a viable and important tool for acquiring datasets in different environments, but it mostly targeted rigid and soft body systems. There is currently dearth of work for more complex fluid systems interacting with immersed solids that can be efficiently and accurately simulated for robot training purposes. In this paper, we propose a new platform called "FishGym", which can be used to train fishlike underwater robots. The framework consists of a robotic fish modeling module using articulated body with skinning, a GPU-based high-performance localized two-way coupled fluidstructure interaction simulation module that handles both finite and infinitely large domains, as well as a reinforcement learning module. We leveraged existing training methods with adaptations to underwater fish-like robots and obtained learned control policies for multiple benchmark tasks. The training results are demonstrated with reasonable motion trajectories, with comparisons and analyses to empirical models as well as known real fish swimming behaviors to highlight the advantages of the proposed platform.

A Survey on Reinforcement Learning Methods in Bionic Underwater Robots

“…However, many of the proposed methods are difficult to compare directly, and the pros and cons of the algorithms are difficult to discern. In 2022, a novel RL training platform FishGym was designed, as reported in [ 139 ], based on a localized, two-way coupled fluid–structure interaction simulation model, and equipped with reinforcement learning components. Inspired by this, if bionic underwater robot platforms can be precisely established as Gym environments through digital twin technology, researchers in the field of reinforcement learning can become more specialized in obtaining higher performances of RL algorithms, laying a foundation for further research on RL intelligent algorithms.…”

Aquarium: A Fully Differentiable Fluid-Structure Interaction Solver for Robotics Applications

“…Finally, Liu et al [38] implement an FSI extension based on the OpenAI Gym environment. Though not differentiable, their fluid solver implementation, a GPU-optimized lattice-Boltzmann method, is highly efficient and can be integrated in an RL training pipeline.…”

“…In the fluids community, various computational fluid dynamics (CFD) solvers also exist [21], [33], [34], [35], [36], [37], [38], [39], [40]. However, these solvers suffer from various deficiencies for use in robotics, including: poor accuracy, generalizability, computational efficiency, and stability over larger time-steps (fully-implicit integration); lack of full differentiability for unsteady flow; and inability to handle fluid-structure interaction (FSI) to properly simulate unsteady, multi-physics coupling between the fluid and solid (e.g., robot) dynamics.…”