Deep reinforcement learning based path stretch vector resolution in dense traffic with uncertainties

Delahaye, Daniel; Pham, Duc-Thinh; Tran, P.; Alam, Sameer; Dương, Vũ

doi:10.1016/j.trc.2021.103463

Cited by 31 publications

(22 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this work, an AI agent is developed to mitigate conflicts, while also minimizing the delay of the aircraft in reaching their metering fixes. It was also shown later in [28] that an AI agent can resolve randomly generated conflict scenarios for a pair of aircraft by prescribing a vectoring maneuver for the ownship to implement. Both of these approaches fail to handle state space scalability as the number of intruder aircraft increase due to the single-agent architecture.…”

Section: B Related Workmentioning

confidence: 98%

A Deep Multi-Agent Reinforcement Learning Approach to Autonomous Separation Assurance

Brittain,

Yang,

Wei

2020

Preprint

View full text Add to dashboard Cite

A novel deep multi-agent reinforcement learning framework is proposed to identify and resolve conflicts among a variable number of aircraft in a high-density, stochastic, and dynamic sector in en route airspace. Currently the sector capacity is limited by human air traffic controller's cognitive limitation. In order to scale up to a high-density airspace, in this work we investigate the feasibility of a new concept (autonomous separation assurance) and a new approach (multi-agent reinforcement learning) to push the sector capacity above human cognitive limitation. We propose the concept of using distributed vehicle autonomy to ensure separation, instead of a centralized sector air traffic controller. Our proposed framework utilizes an actor-critic model, Proximal Policy Optimization (PPO) that we customize to incorporate an attention network. By using the attention network, we are able to encode the information from a variable number of intruder aircraft into a fixed length vector and allow the agents to learn which intruder aircraft's information is critical to achieve the optimal performance. This allows the agents to have access to variable aircraft information in the sector in a scalable, efficient approach to achieve high traffic throughput under uncertainty. The agents are trained using a centralized learning, decentralized execution scheme where one neural network is learned and shared by all agents in the environment. To validate the proposed framework, we designed three challenging case studies in the BlueSky air traffic control environment. Numerical results show the proposed framework significantly reduces the offline training time without sacrificing performance.

show abstract

Section: B Related Workmentioning

confidence: 98%

A Deep Multi-Agent Reinforcement Learning Approach to Autonomous Separation Assurance

Brittain,

Yang,

Wei

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The time at CPA is the time at which the smallest distance between the two aircraft occurs. To compute the CPA at the horizontal axis we follow the methodology presented in [13].…”

Section: Definitionsmentioning

confidence: 99%

“…Motivated by the bibliography on CD&R ( [13], [14], [15], [16]), we enrich trajectory states with a vector v of variables including, the magnitudes of the aircraft horizontal (s h ) and vertical (s v ) speed, as well as with the following features regarding any neighboring trajectory T j :…”

Section: Trajectory States Atco Modes Of Reaction and Resolution Actionsmentioning

confidence: 99%

“…To compute the CPA we follow the methodology presented in [13]. As we want our model to be agnostic to the specific spatio-temporal (longitude, latitude, timestamp) position, so as to generalize beyond specific areas and origin destination pairs, we train and test our models using an abstraction of states as follows:…”

Section: Trajectory States Atco Modes Of Reaction and Resolution Actionsmentioning

confidence: 99%

“…The application of Reinforcement Learning (RL) methods on the CD&R task has also received a lot of attention. Authors in [14] and [13] resolve conflicts between two aircraft in the presence of uncertainty. While in [16] authors formulate the problem as a multi-agent reinforcement learning problem and propose a Message Passing Actor Critic Model inspired by DGN [24], while also exploiting Message Passing Neural Networks [25].…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Data-driven prediction of Air Traffic Controllers reactions to resolving conflicts

Bastas¹,

Vouros²

2022

Preprint

View full text Add to dashboard Cite

With the aim to enhance automation in conflict detection and resolution (CD&R) tasks in the Air Traffic Management domain, in this paper we propose deep learning techniques (DL) that can learn models of Air Traffic Controllers' (ATCO) reactions in resolving conflicts that can violate separation minimum constraints among aircraft trajectories: This implies learning when the ATCO will react towards resolving a conflict, and how he/she will react. Timely reactions, to which this paper aims, focus on when do reactions happen, aiming to predict the trajectory points, as the trajectory evolves, that the ATCO issues a conflict resolution action, while also predicting the type of resolution action (if any). Towards this goal, the paper formulates the ATCO reactions prediction problem for CD&R, and presents DL methods that can model ATCO timely reactions and evaluates these methods in real-world data sets, showing their efficacy in prediction with very high accuracy.

show abstract

Aircraft Conflict Resolution Using Convolutional Neural Network on Trajectory Image

Rahman

Lapasset

Mothe

2022

Intelligent Systems Design and Applications

View full text Add to dashboard Cite

A situation between several moving aircraft is a conflict when their position is less than the internationally specified distance. To solve aircraft conflicts, air traffic controllers consider many parameters including the positioning coordinate, speed, direction, weather, etc. of the involved aircraft. This is a complex task, specifically considering the increase of the traffic. Assisting systems could help controllers in their tasks. Most conflict resolution models are based on trajectory data of a fixed number of input aircraft. Under this constraint, it is possible to resolve conflicts using machine learning models, including convolutional neuron network models. Such models cannot resolve conflicts that imply a variable number of aircraft because the input size of the model is fixed. To solve this challenge, we transformed the trajectory data into images which size does not depend on the number of planes. We developed a multi-label conflict resolution model that we named ACRnet, based on a convolutional neural network to classify the obtained images. ACRnet model achieves an accuracy of 99.16% on the training data and of 98.97% on the test data set for two aircraft. For both two and three aircraft, the accuracy is 99.05% (resp. 98.96%) on the training (resp. test) data set.

show abstract

Deep reinforcement learning based path stretch vector resolution in dense traffic with uncertainties

Cited by 31 publications

References 16 publications

A Deep Multi-Agent Reinforcement Learning Approach to Autonomous Separation Assurance

A Deep Multi-Agent Reinforcement Learning Approach to Autonomous Separation Assurance

Data-driven prediction of Air Traffic Controllers reactions to resolving conflicts

Aircraft Conflict Resolution Using Convolutional Neural Network on Trajectory Image

Contact Info

Product

Resources

About