Physics Informed Deep Reinforcement Learning for Aircraft Conflict Resolution

Liu, Yongming

doi:10.1109/tits.2021.3077572

Cited by 30 publications

(18 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Brittain and Wei in [5] applied two-level deep reinforcement learning in a hierarchical way while Pham et al [14,15] proposed a single agent-based deep reinforcement learning model for two aircraft at the same altitude. Zhao and Liu [21] also applied reinforcement learning and CNN using image data where each image contains the current position of aircraft associated with the conflict. However, a proper reward function is required for the best resolution of reinforcement learning and finding that function is the most challenging task because it depends on features like certain rules and conditions.…”

Section: Related Workmentioning

confidence: 99%

“…By combining both ideas of 5-minute trajectory and image data [18,21], we suggest here a multi-label classification model based on a CNN where each image contains a 5-minute trajectory for each aircraft associated with the conflict. The main purpose of this research is not to find the best image processing model but to easily overcome many challenges of existing sequence-based models through image processing with higher performance to resolve the aircraft conflict.…”

Section: Related Workmentioning

confidence: 99%

“…The resolution of conflicts where a variable number of aircraft are involved however cannot be handled and remains an important challenge. Brittain and Wei [6] applied long short-term memory (LSTM) and Zhao and Liu [21] convert trajectory data to image data to handle a variable number of aircraft in real-time using reinforcement learning. The computation of their model depends on the number of aircraft when using LSTM, and finding the perfect reward function when using reinforcement is challenging.…”

Section: Introductionmentioning

confidence: 99%

“…The computation of their model depends on the number of aircraft when using LSTM, and finding the perfect reward function when using reinforcement is challenging. Here, inheriting ideas from Zhao and Liu [21], we convert trajectory data to image data. The benefits of converting trajectory data to images are given below:…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Aircraft Conflict Resolution Using Convolutional Neural Network on Trajectory Image

Rahman

Lapasset

Mothe

2022

Intelligent Systems Design and Applications

View full text Add to dashboard Cite

A situation between several moving aircraft is a conflict when their position is less than the internationally specified distance. To solve aircraft conflicts, air traffic controllers consider many parameters including the positioning coordinate, speed, direction, weather, etc. of the involved aircraft. This is a complex task, specifically considering the increase of the traffic. Assisting systems could help controllers in their tasks. Most conflict resolution models are based on trajectory data of a fixed number of input aircraft. Under this constraint, it is possible to resolve conflicts using machine learning models, including convolutional neuron network models. Such models cannot resolve conflicts that imply a variable number of aircraft because the input size of the model is fixed. To solve this challenge, we transformed the trajectory data into images which size does not depend on the number of planes. We developed a multi-label conflict resolution model that we named ACRnet, based on a convolutional neural network to classify the obtained images. ACRnet model achieves an accuracy of 99.16% on the training data and of 98.97% on the test data set for two aircraft. For both two and three aircraft, the accuracy is 99.05% (resp. 98.96%) on the training (resp. test) data set.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Aircraft Conflict Resolution Using Convolutional Neural Network on Trajectory Image

Rahman

Lapasset

Mothe

2022

Intelligent Systems Design and Applications

View full text Add to dashboard Cite

show abstract

“…As a result, encoding physical laws in machine learning models essentially reduces the number of training points that are required to tune a deep learning model. The benefits of exploiting physical laws in building efficient deep learning models have been showcased in several recent studies [34,35,36,37,38,39]. It is worthwhile noting that the loss functions of PINNs is complicated and involve multiple terms, which would compete with each other during the training process [32].…”

Section: Introductionmentioning

confidence: 99%

A generic physics-informed neural network-based framework for reliability assessment of multi-state systems

Zhou¹,

Droguett²,

Mosleh³

2021

Preprint

View full text Add to dashboard Cite

In this paper, we leverage the recent advances in physics-informed neural network (PINN) and develop a generic PINN-based framework to assess the reliability of multi-state systems (MSSs). The proposed methodology consists of two major steps. In the first step, we recast the reliability assessment of MSS as a machine learning problem using the framework of PINN. A feedforward neural network with two individual loss groups are constructed to encode the initial condition and state transitions governed by ordinary differential equations (ODEs) in MSS. Next, we tackle the problem of high imbalance in the magnitude of the back-propagated gradients in PINN from a multi-task learning perspective. Particularly, we treat each element in the loss function as an individual task, and adopt a gradient surgery approach named projecting conflicting gradients (PCGrad), where a task's gradient is projected onto the norm plane of any other task that has a conflicting gradient. The gradient projection operation significantly mitigates the detrimental effects caused by the gradient interference when training PINN, thus accelerating the convergence speed of PINN to high-precision solutions to MSS reliability assessment. With the proposed PINN-based framework, we investigate its applications for MSS reliability assessment in several different contexts in terms of time-independent or dependent state transitions and system scales varying from small to medium. The results demonstrate that the proposed PINN-based framework shows generic and remarkable performance in MSS reliability assessment, and the incorporation of PCGrad in PINN leads to substantial improvement in solution quality and convergence speed.

show abstract

Locally generalised multi-agent reinforcement learning for demand and capacity balancing with customised neural networks

CHEN

et al. 2023

Chinese Journal of Aeronautics

View full text Add to dashboard Cite

Physics Informed Deep Reinforcement Learning for Aircraft Conflict Resolution

Cited by 30 publications

References 22 publications

Aircraft Conflict Resolution Using Convolutional Neural Network on Trajectory Image

Aircraft Conflict Resolution Using Convolutional Neural Network on Trajectory Image

A generic physics-informed neural network-based framework for reliability assessment of multi-state systems

Locally generalised multi-agent reinforcement learning for demand and capacity balancing with customised neural networks

Contact Info

Product

Resources

About