Reward functions for learning to control in air traffic flow management

Cruciol, Leonardo L. B. V.; Arruda, Antonio Carlos de; Weigang, Li; Li, Leihong; Crespo, António

doi:10.1016/j.trc.2013.06.010

Cited by 45 publications

(26 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiple works that have applied reinforcement learning within air traffic control define aircraft as agents [44][45][46][47][48]. However, for air traffic control flow, preference for defining the agent is often given to some structural element within the operational environment [49]. This allows for a general control over aircraft, without having to directly control each single aircraft.…”

Section: Agentmentioning

confidence: 99%

Velocity Obstacle Based Conflict Avoidance in Urban Environment with Variable Speed Limit

Ribeiro

Ellerbroek

2021

Aerospace

View full text Add to dashboard Cite

Current investigations into urban aerial mobility, as well as the continuing growth of global air transportation, have renewed interest in conflict detection and resolution (CD&R) methods. The use of drones for applications such as package delivery, would result in traffic densities that are orders of magnitude higher than those currently observed in manned aviation. Such densities do not only make automated conflict detection and resolution a necessity, but will also force a re-evaluation of aspects such as coordination vs. priority, or state vs. intent. This paper looks into enabling a safe introduction of drones into urban airspace by setting travelling rules in the operating airspace which benefit tactical conflict resolution. First, conflicts resulting from changes of direction are added to conflict resolution with intent trajectory propagation. Second, the likelihood of aircraft with opposing headings meeting in conflict is reduced by separating traffic into different layers per heading–altitude rules. Guidelines are set in place to make sure aircraft respect the heading ranges allowed at every crossed layer. Finally, we use a reinforcement learning agent to implement variable speed limits towards creating a more homogeneous traffic situation between cruising and climbing/descending aircraft. The effects of all of these variables were tested through fast-time simulations on an open source airspace simulation platform. Results showed that we were able to improve the operational safety of several scenarios.

show abstract

Section: Agentmentioning

confidence: 99%

Velocity Obstacle Based Conflict Avoidance in Urban Environment with Variable Speed Limit

Ribeiro

Ellerbroek

2021

Aerospace

View full text Add to dashboard Cite

show abstract

“…It was tested using a simulation tool FACET (Future ATM Concept Evaluation Tool). Based on this research, [7] further enriched the RL reward function by considering the safety factors and fairness impact. Case studies in Brazil were described to show the effectiveness and efficiency of the new RL reward functions in the controller decision process of ATFM.…”

Section: Previous Studymentioning

confidence: 99%

Policy Optimization in Automated Point Merge Trajectory Planning: An Artificial Intelligence-based Approach

Liang¹,

Weigang²,

Delahaye³

et al. 2019

2019 IEEE/AIAA 38th Digital Avionics Systems Conference (DASC)

View full text Add to dashboard Cite

Air traffic management is a complex decision making process. Air traffic controllers decision on aircraft trajectory control actions directly lead to the efficiency of traffic flow management. This paper aims to realize an automated routine trajectory management in terminal manoeuvring area with an intelligent decision making agent. An artificial intelligence based approach is applied to adaptively and smartly integrate four types of deconflict actions for resolving conflicts. Especially, the reinforcement learning policy optimization process is discussed in detail. Firstly, application of reinforcement learning in adaptive trajectory planning is presented. The entire problem is adaptively divided into several sub-problems. For each sub-problem, an online policy is applied to guide the simulation and optimization modules to find out the conflict free and less delay solution. The online policy is a scale of weight distribution for choosing desirable actions. It follows the rule of roulette wheel selection with weighted probability. The highest desirable decision variable has the largest share of the roulette wheel, while the lowest desirable decision variable has the smallest share of the roulette wheel. Direct policy optimization algorithm is designed to update the online policy. Finally, experiments are built up for validation of the proposed policy optimization algorithm for the intelligent decision making process. The results in the test environment showed that learning agent with different exploration and exploitation ability will result in different system performance in conflict resolution and delay.

show abstract

“…As such, an excessive amount of aircraft in one sector affects the overall safety of the system. The function Cs(x) gives the severity of congestion in each sector x [15]:…”

Section: A Preference Functions For Air Traffic Controlmentioning

confidence: 99%

Satisficing Collaborative Decision Making and controlling for airport management

Almeida

Weigang

Meinerz³

2014

2014 IEEE/ASME 10th International Conference on Mechatronic and Embedded Systems and Applications (MESA)

View full text Add to dashboard Cite

The operations of takeoff and landing at airports are often subject to some delays caused by the application of the Ground Delay Program (GDP) and Air Holding Program (AHP). The effects of these delays impact to the related entities in the traffic scenario, such as Air Traffic Control (ATC) unity and airlines. As a result, Collaborative Decision Making (CDM) is being employed as a methodology for increasing the efficiency of air traffic management, through the involvement of partners within the airports. However, many current implementations of CDM are focused on the sharing of information. This study proposes a CDM model based on Satisficing Game Theory for computing decisions involving the three main stakeholders in airport scenario: airport service management, airlines, and ATC unity. Initially, we defined the preference for each entity, such as rejectability, and selectability functions. We then built the Satisficing CDM model to establish the collaborative decision process in airport. The case study shows the effective of the developed system in the air terminal area to define the sequences of the flights in take-off and landing.

show abstract

Reward functions for learning to control in air traffic flow management

Cited by 45 publications

References 17 publications

Velocity Obstacle Based Conflict Avoidance in Urban Environment with Variable Speed Limit

Velocity Obstacle Based Conflict Avoidance in Urban Environment with Variable Speed Limit

Policy Optimization in Automated Point Merge Trajectory Planning: An Artificial Intelligence-based Approach

Satisficing Collaborative Decision Making and controlling for airport management

Contact Info

Product

Resources

About