Autonomous Thermalling as a Partially Observable Markov Decision Process

Guilliard, Iain; Rogahn, Richard; Piavi, Jim; Kolobov, Andrey

doi:10.15607/rss.2018.xiv.068

Cited by 14 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reddy et al [15], like Wharington [18], present a proof-of-concept approach in simulation, but utilize the much more detailed thermal simulation based on Rayleigh-Bénard convection. Guilliard et al [10] compare ArduSoar to another thermalling controller, POMDSoar, that generates thermalling trajectories by approximately solving a POMDP and thereby explicitly plans for thermal exploration. In turbulent conditions of Guilliard et al [10]'s experiments POMDSoar outperformed ArduSoar, but critically relied on accurate estimates of several additional airframe-specific parameters to do well, making POMDSoar more involved to use in practice.…”

Section: Related Workmentioning

confidence: 99%

ArduSoar: an Open-Source Thermalling Controller for Resource-Constrained Autopilots

Tabor¹,

Guilliard²,

Kolobov³

2018

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomous soaring capability has the potential to significantly increase time aloft for fixed-wing UAVs. In this paper, we introduce ArduSoar, the first soaring controller integrated into a major autopilot software suite for small UAVs. We describe ArduSoar from the algorithmic standpoint, outline its integration with the ArduPlane autopilot, discuss parameter tuning for it, and conduct a series of flight tests on real sUAVs that show ArduSoar's robustness even in highly nonideal atmospheric conditions.

show abstract

Section: Related Workmentioning

confidence: 99%

ArduSoar: an Open-Source Thermalling Controller for Resource-Constrained Autopilots

Tabor¹,

Guilliard²,

Kolobov³

2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The other common autonomous soaring strategies include mechanical energy-based feedback control and the extended Kalman filter-based method (Kahn, 2017; Notter et al , 2020). Furthermore, the Markov Decision Process had been introduced in the autopilot to build a simulator for a specific thermal (Guilliard et al , 2018).…”

Section: Introductionmentioning

confidence: 99%

Intelligent soaring and path planning for solar-powered unmanned aerial vehicles

Wu,

Wen,

Zhao

et al. 2024

AEAT

View full text Add to dashboard Cite

Purpose This study aims to study the thermal identification issue by harvesting both solar energy and atmospheric thermal updraft for a solar-powered unmanned aerial vehicle (SUAV) and its electric energy performance under continuous soaring conditions. Design/methodology/approach The authors develop a specific dynamic model for SUAVs in both soaring and cruise modes. The support vector machine regression (SVMR) is adopted to estimate the thermal position, and it is combined with feedback control to implement the SUAV soaring in the updraft. Then, the optimal path model is built based on the graph theory considering the existence of several thermals distributed in the environment. The procedure is proposed to estimate the electricity cost of SUAV during flight as well as soaring, and making use of dynamic programming to maximize electric energy. Findings The simulation results present the integrated control method could allow SUAV to soar with the updraft. In addition, the proposed approach allows the SUAV to fly to the destination using distributed thermals while reducing the electric energy use. Originality/value Two simplified dynamic models are constructed for simulation considering there are different flight mode. Besides, the data-driven-based SVMR method is proposed to support SUAV soaring. Furthermore, instead of using length, the energy cost coefficient in optimization problem is set as electric power, which is more suitable for SUAV because its advantage is to transfer the three-dimensional path planning problem into the two-dimensional.

show abstract

“…Experimental tests using high-fidelity numerical simulations of real wind fields showed that the energy map method achieved good results in solving the problem of long-distance flight in unmanned aerial vehicles. Guilliard et al [13] dealt with the problem of the autonomous balanced utilization of updrafts and environmental mapping as a partially observable MDP, which was based on predicted trajectories when given actual system states rather than generating Markov tuples offline. The researchers further applied this algorithm to the popular open-source autopilot ArduPlane and compared it with existing alternative algorithms in a series of real-time flight experiments.…”

Section: Introductionmentioning

confidence: 99%

Energy-Harvesting Strategy Investigation for Glider Autonomous Soaring Using Reinforcement Learning

Zhao,

Li,

Zeng

2023

Aerospace

View full text Add to dashboard Cite

Birds and experienced glider pilots frequently use atmospheric updrafts for long-distance flight and energy conservation, with harvested energy from updrafts serving as the foundation. Inspired by their common characteristics in autonomous soaring, a reinforcement learning algorithm, the Twin Delayed Deep Deterministic policy gradient, is used to investigate the optimal strategy for an unpowered glider to harvest energy from thermal updrafts. A round updraft model is utilized to characterize updrafts with varied strengths. A high-fidelity six-degree-of-glider model is used in the dynamic modeling of a glider. The results for various flight initial positions and updraft strengths demonstrate the effectiveness of the strategy learned via reinforcement learning. To enhance the updraft perception ability and expand the applicability of the trained glider agent, an extra wind velocity differential correction module is introduced to the algorithm, and a strategy symmetry method is applied. Comparison experiments regarding round updraft, the Gedeon thermal model, and Dryden continuous turbulence indicate the crucial role of the further optimized methods in improving the updraft-sensing ability of the reinforcement learning glider agent. With optimized methods, a glider trained in a simplified thermal updraft with a simple training method can have more effective flight strategies.

show abstract

Autonomous Thermalling as a Partially Observable Markov Decision Process

Cited by 14 publications

References 27 publications

ArduSoar: an Open-Source Thermalling Controller for Resource-Constrained Autopilots

ArduSoar: an Open-Source Thermalling Controller for Resource-Constrained Autopilots

Intelligent soaring and path planning for solar-powered unmanned aerial vehicles

Energy-Harvesting Strategy Investigation for Glider Autonomous Soaring Using Reinforcement Learning

Contact Info

Product

Resources

About