Reward Shaping with Dynamic Trajectory Aggregation

Okudo, Takato; Yamada, Seiji

doi:10.1109/ijcnn52387.2021.9533401

Cited by 1 publication

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, the reduced-order models and learning algorithms used in this study have potential improvements that can be made in future research, including (1) collecting more quantitative data on storm petrel pattering/seaanchoring to validate modeling approaches, (2) exploring new learning techniques, such as reward shaping [37,38], to enhance the interoperability of the AI 'black box' , and (3) using empirical data to measure the drag coefficient of objects that replicate a storm petrel's anatomy by fluid dynamic experiments to build a higher fidelity force model [39][40][41].…”

Section: Discussionmentioning

confidence: 99%

Exploring storm petrel pattering and sea-anchoring using deep reinforcement learning

Xue,

Han,

Klaassen van Oorschot

et al. 2023

Bioinspir. Biomim.

View full text Add to dashboard Cite

Developing hybrid aerial-aquatic vehicles that can interact with water surfaces while remaining aloft is valuable for various tasks, including ecological monitoring, water quality sampling, and search and rescue operations. Storm petrels are a group of pelagic seabirds that exhibit a unique locomotion pattern known as "pattering" or "sea-anchoring," which is hypothesized to support forward locomotion and/or stationary posture at the water surface. In this study, we use morphological measurements of three storm petrel species and aero/hydrodynamic models to develop a computational storm petrel model and interact it with a hybrid fluid environment. Using Deep Reinforcement Learning (DRL) algorithms, we find that the storm petrel model exhibits high maneuverability and stability under a wide range of constant wind velocities after training. We also verify in the simulation that the storm petrel can use its “pattering” or “sea-anchoring” behavior to achieve different biomechanical sub-tasks (e.g., weight support, forward locomotion, stabilization) and adapt it under different wind speeds and optimization objectives. Specifically, we observe an adjustment in storm petrel’s movement patterns as wind velocity increases and quantitively analyze its biomechanics underneath. Our results provide new insights into how storm petrels achieve efficient locomotion and dynamic stability at the air-water interface and adapt their behaviors to different wind velocities and tasks in open environments. Ultimately, our study will guide the design of next-generation biomimetic petrel-inspired robots for tasks requiring proximity to the water interface and efficiency.

show abstract