Learning Based Speed Control of Soft Robotic Fish

Abstract

Bioinspired robotics takes advantage of biological systems in nature for morphology, action and perception to build advanced robots of compelling performance and wide application. This paper focuses on the design, modeling and control of a bioinspired robotic fish. The design utilizes a recently-developed artificial muscle named super coiled polymer for actuation and a soft material (silicone rubber) for building the robot body. The paper proposes a learning based speed control design approach for bioinspired robotic fish using model-free reinforcement learning. Based on a mathematically tractable dynamic model derived by approximating the robotic fish with a three-link robot, speed control simulation is conducted to demonstrate and validate the control design method. Exampled with a three-link reduced-order dynamic system, the proposed learning based control design approach is applicable to many and various complicated bioinspired robotic systems.

Design, Modeling, and Visual Learning-Based Control of Soft Robotic Fish Driven by Super-Coiled Polymers

“…SCP, a recently developed artificial muscle actuator, is lightweight, flexible, strong with a high power-toweight ratio and fabricated with silver-plated nylon threads (Yip and Niemeyer, 2017). Our study also showed through simulation that speed control of a one-dimensional robotic fish was successfully done with SCP actuators using reinforcement learning (Rajendran and Zhang, 2018;Sutton and Barto, 2018). Nevertheless, besides employing a sparsely discretized state space in the dynamics, our previous model is dimensionally limited which is too simplified to mimic the biological fish and study the swimming motion.…”

“…Following this, aiming towards a phased approach at developing reinforcement learning-based control for the soft robotic fish, a foundational Q-learning (Watkins and Dayan, 1992) based controller was designed and simulated to control the speed of a three-link robotic fish which consisted of discretized state and action spaces (Rajendran and Zhang, 2018). The robotic fish was restricted to one-dimensional locomotion and the agent was trained until the Frobenius norm between the current and previous Q-tables was minimized to a threshold.…”

“…Taking advantage of natural biological structures, functions, and motions of aquatic animals aids us in creating underwater robots which are energy and locomotion efficient, and possess agile maneuverability, for a diverse range of purposes. Our research focuses on developing a biomimetic underwater soft robotic fish that can self-learn its locomotion to achieve different goals such as regulating its angle of orientation and adapting to variable swimming speeds (Rajendran and Zhang, 2018), which eventually serve as decomposed control tasks for high-level control objectives such as traversing along a planned trajectory and studying fish swarming behavior like schooling and shoaling.…”

“…Traditional actuators such as electrical motors and pneumatic/ hydraulic cylinders which are employed to realize fish undulations in the aforementioned multi-link robotic fish prototypes, although offer a high output force/torque, are generally heavy and quite rigid, thus making fish robots less flexible. Hence, the use of soft actuators such as artificial muscles like pneumatic artificial muscles (PAM), ionic polymer-metal composites (IPMC) (Chen, 2017;Olsen and Kim, 2019), dielectric elastomer actuators (Christianson et al, 2019), and super-coiled polymers (SCP) (Yip and Niemeyer, 2017;Rajendran and Zhang, 2018;Simeonov et al, 2018) is on the rise. Not only are artificial muscles slender, but also strong, flexible, lightweight, and analogously compliant to biological muscles.…”

“…As several robotic fish prototypes adopt various closed-loop control techniques such as PID control (Yu et al, 2004;Berlinger et al, 2021), PI control (Zhang et al, 2015a), central pattern generator control (Jeong et al, 2011), pre-trained neural networks (Thuruthel et al, 2019), robust control (Zhang et al, 2015b), to improve the performance of locomotion, others employ open-loop control techniques whereby a predefined swimming profile is generated to perform a coded set of actions (lookup table) which is predominantly used in cases of complex or highly nonlinear robotic fish dynamic models (Yu and Wang, 2005;Korkmaz et al, 2012). However, in order to address the problems of high nonlinearity and intrinsically infinite system dimension, researchers are looking into various present-day techniques in artificial intelligence (Rajendran and Zhang, 2018;Bhagat et al, 2019;Thuruthel et al, 2019), more specifically behavior-based or adaptive machine learning-based control.…”