A numerical simulation of target-directed swimming for a three-link bionic fish with deep reinforcement learning

Abstract

An accurate and robust feedback control system is of great importance for bionic robotic fish to perform complex tasks. This work presents a numerical study of target-directed swimming for a three-link bionic fish with a feedback control system based on deep reinforcement learning (DRL). The simulation is achieved by using a hybrid method of the DRL method and the immersed boundary–lattice Boltzmann method (IB–LBM). This framework makes use of the high computational efficiency of the IB-LBM for the generation of massive motion data needed in the DRL training. The fish is first trained to swim towards a static target from random orientation and distance. The only information available to the fish is its orientation and distance from the target. It learns an accurate and subtle control policy after the training. Then the control policy is applied to a moving target. Even though the fish encounters some situations that never happened in the training, it can choose appropriate actions to follow the target in close proximity and automatically adapt its velocity with the velocity of the target. Those simulations demonstrate the accuracy, robustness of the control method, and its ability to adapt to new situations.

Learning obstacle avoidance and predation in complex reef environments with deep reinforcement learning

“…The distance outcome-based reward r 2 and time constraint reward r 1 stand out as the sole rewards directly linked to the ultimate objective, having been the central focus of reward configurations in prior investigations [25]. Their objective is to incentivize the fish to minimize the distance to the target expeditiously.…”

“…In addressing the stationary food fixed-point navigation problem, employing a value-based RL algorithm with a single distance outcome-based reward r 2 and a time constraint reward r 1 can effectively achieve the task [25]. In the case of food exhibiting random movement, when the intelligent fish approaches the target closely, the food's random movement can result in a sudden increase in the distance between the fish and the food, with a high probability.…”

The lattice Boltzmann method and its applications in engineering flows

“…Han and Chen 4 study the unsteady hydrodynamics of flapping foils in schooling transitions between tandem and diamond schooling configurations with an immersed boundary-lattice Boltzmann method, identifying an optimal energy-saving transition mode: clockwise arc mode. Zhu and Pang 5 apply the hybrid frame work of a multi-block geometry-adaptive immersed boundary-lattice Boltzmann method and the deep reinforcement learning in modelling target-directed swimming of a three-link bionic fish, finding that the trained fish can make appropriate decisions to reach the target even with new situations never encountered in the training stage. Wang et al 6 review the recent progress of LBM and its applications, particularly focusing on bio-inspired flows involving fluid-structure interaction (e.g.…”