Learning legged swimming gaits from experience

Abstract

We present an end-to-end framework for realizing fully automated gait learning for a complex underwater legged robot. Using this framework, we demonstrate that a hexapod flipper-propelled robot can learn task-specific control policies purely from experience data. Our method couples a state-of-theart policy search technique with a family of periodic low-level controls that are well suited for underwater propulsion. We demonstrate the practical efficacy of tabula rasa learning, that is, learning without the use of any prior knowledge, of policies for a six-legged swimmer to carry out a variety of acrobatic maneuvers in three dimensional space. We also demonstrate informed learning that relies on simulated experience from a realistic simulator. In numerous cases, novel emergent gait behaviors have arisen from learning, such as the use of one stationary flipper to create drag while another oscillates to create thrust. Similar effective results have been demonstrated in under-actuated configurations, where as few as two flippers are used to maneuver the robot to a desired pose, or through an acrobatic motion such as a corkscrew. The success of our learning framework is assessed both in simulation and in the field using an underwater swimming robot.

Robotic Coral Reef Health Assessment Using Automated Image Analysis

“…We have recently demonstrated that the robot can learn an even wider variety of complex motions by learning autonomously from repeated trials. The quality of the learned motion controllers are comparable with those set by an expert human engineer (Meger et al., ). While these learned behaviors were not exploited in the data collection runs used in this paper, we foresee using them in the near future to better exploit the acquired visual data.…”

“…(Giguere, Girdhar, & Dudek, ) note that gain scheduling is needed for stable control of a swimming robot across multiple speeds because of nonlinear drag effects. In prior work with the Aqua class of vehicles developed in our lab, we have demonstrated a combination of small size, low weight, and high maneuverability with diverse gaits (Meger, Higuera, Xu, & Dudek, ; Meger, Shkurti, Cortes Poza, Giguere, & Dudek, ). In (Giguere et al., ), we have developed a controller to allow our vehicle to autonomously move over coral reef structures using visual feedback.…”

Efficient reinforcement learning for robots using informative simulated priors

“…Mixing simulated and real data has been shown to cause poor performance as the GP models of the realworld transition dynamics can become corrupted by incorrect simulation data [23]. In our approach, even with an incorrect simulator, real data from the target domain will eventually overcome the effects of the prior and converge to the true transition dynamics as the number of obtained data points increases.…”

Design and Modelling of an Amphibious Spherical Robot Attached with Assistant Fins

“…Another type of amphibious robots is equipped with flippers that act as paddles in water and as legs on land. Using six paddles for propulsion, AQUA is suitable for navigating in shallow-water environment [12,13]. With the help of an acoustic-based localization system and a vision-based localization system, AQUA obtained the ability to revisit a previously visited site autonomously [14,15].…”