Learning to Perform a Perched Landing on the Ground Using Deep Reinforcement Learning

Waldock, Antony; Greatwood, Colin; Salama, Francis; Richardson, Thomas

doi:10.1007/s10846-017-0696-1

Cited by 35 publications

(34 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some RL methods have been proposed to solve aircraft guidance problems. [18][19][20][21][22] An RL agent is developed for guiding a powered UAV from one thermal location to another quickly and efficiently by controlling bank angle 18 in the x-y plane. To solve guidance problem with many conflicting objectives, a voting Q-learning algorithm is proposed.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A pretrained proximal policy optimization algorithm with reward shaping for aircraft guidance to a moving destination in three-dimensional continuous space

Wang

et al. 2021

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

To enhance the performance of guiding an aircraft to a moving destination in a certain direction in three-dimensional continuous space, it is essential to develop an efficient intelligent algorithm. In this article, a pretrained proximal policy optimization (PPO) with reward shaping algorithm, which does not require an accurate model, is proposed to solve the guidance problem of manned aircraft and unmanned aerial vehicles. Continuous action reward function and position reward function are presented, by which the training speed is increased and the performance of the generated trajectory is improved. Using pretrained PPO, a new agent can be trained efficiently for a new task. A reinforcement learning framework is built, in which an agent can be trained to generate a reference trajectory or a series of guidance instructions. General simulation results show that the proposed method can significantly improve the training efficiency and trajectory performance. The carrier-based aircraft approach simulation is carried out to prove the application value of the proposed approach.

show abstract

Section: Introductionmentioning

confidence: 99%

“…A DQN algorithm is used to generate a trajectory to perform a perched landing on the ground. 22 In this DQN algorithm, noise is added to the numerical model of airspeed in the training environment, which is more in line with the actual scenario.…”

Section: Introductionmentioning

confidence: 99%

A pretrained proximal policy optimization algorithm with reward shaping for aircraft guidance to a moving destination in three-dimensional continuous space

Wang

et al. 2021

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

show abstract

“…A similar approach was proposed in another study [ 13 ], which used DRL to train the aircraft by heading commands and constant speed to guide the aircraft. A trajectory generating method was proposed by using a DQN algorithm to perform a perched landing on the ground [ 14 ]. In the above DQN algorithm, noise is considered by the model, which is more in line with the actual scenario in the training process.…”

Section: Introductionmentioning

confidence: 99%

A Multi-Dimensional Goal Aircraft Guidance Approach Based on Reinforcement Learning with a Reward Shaping Algorithm

Yang

Liu

et al. 2021

Sensors

View full text Add to dashboard Cite

Guiding an aircraft to 4D waypoints at a certain heading is a multi-dimensional goal aircraft guidance problem. [d=Zu]In order to improve the performance and solve this problem, this paper proposes a multi-layer RL approach.To enhance the performance, in the present study, a multi-layer RL approach to solve the multi-dimensional goal aircraft guidance problem is proposed. The approach [d=Zu]enablesassists the autopilot in an ATC simulator to guide an aircraft to 4D waypoints at certain latitude, longitude, altitude, heading, and arrival time, respectively. To be specific, a multi-layer RL [d=Zu]approach is proposedmethod to simplify the neural network structure and reduce the state dimensions. A shaped reward function that involves the potential function and Dubins path method is applied. [d=Zu]Experimental and simulation results show that the proposed approachExperiments are conducted and the simulation results reveal that the proposed method can significantly improve the convergence efficiency and trajectory performance. [d=Zu]FurthermoreFurther, the results indicate possible application prospects in team aircraft guidance tasks, since the aircraft can directly approach a goal without waiting in a specific pattern, thereby overcoming the problem of current ATC simulators.

show abstract

“…DRL has been utilized in many decision-making fields, such as video games [26,27], board games [28,29], and robot control [30] and obtained great achievements of human level or superhuman performance. For aircraft guidance research, Waldock et al [31] proposed a DQN method to generate a trajectory to perform perched landing on the ground. Alejandro et al [32] proposed a DRL strategy for autonomous landing of the UAV on a moving platform.…”

Section: Introductionmentioning

confidence: 99%

Improving Maneuver Strategy in Air Combat by Alternate Freeze Games with a Deep Reinforcement Learning Algorithm

Wang

et al. 2020

Mathematical Problems in Engineering

View full text Add to dashboard Cite

In a one-on-one air combat game, the opponent’s maneuver strategy is usually not deterministic, which leads us to consider a variety of opponent’s strategies when designing our maneuver strategy. In this paper, an alternate freeze game framework based on deep reinforcement learning is proposed to generate the maneuver strategy in an air combat pursuit. The maneuver strategy agents for aircraft guidance of both sides are designed in a flight level with fixed velocity and the one-on-one air combat scenario. Middleware which connects the agents and air combat simulation software is developed to provide a reinforcement learning environment for agent training. A reward shaping approach is used, by which the training speed is increased, and the performance of the generated trajectory is improved. Agents are trained by alternate freeze games with a deep reinforcement algorithm to deal with nonstationarity. A league system is adopted to avoid the red queen effect in the game where both sides implement adaptive strategies. Simulation results show that the proposed approach can be applied to maneuver guidance in air combat, and typical angle fight tactics can be learnt by the deep reinforcement learning agents. For the training of an opponent with the adaptive strategy, the winning rate can reach more than 50%, and the losing rate can be reduced to less than 15%. In a competition with all opponents, the winning rate of the strategic agent selected by the league system is more than 44%, and the probability of not losing is about 75%.

show abstract

Learning to Perform a Perched Landing on the Ground Using Deep Reinforcement Learning

Cited by 35 publications

References 7 publications

A pretrained proximal policy optimization algorithm with reward shaping for aircraft guidance to a moving destination in three-dimensional continuous space

A pretrained proximal policy optimization algorithm with reward shaping for aircraft guidance to a moving destination in three-dimensional continuous space

A Multi-Dimensional Goal Aircraft Guidance Approach Based on Reinforcement Learning with a Reward Shaping Algorithm

Improving Maneuver Strategy in Air Combat by Alternate Freeze Games with a Deep Reinforcement Learning Algorithm

Contact Info

Product

Resources

About