Using Reinforcement Learning to Improve Airspace Structuring in an Urban Environment

Abstract

Current predictions on future drone operations estimate that traffic density orders of magnitude will be higher than any observed in manned aviation. Such densities redirect the focus towards elements that can decrease conflict rate and severity, with special emphasis on airspace structures, an element that has been overlooked within distributed environments in the past. This work delves into the impacts of different airspace structures in multiple traffic scenarios, and how appropriate structures can increase the safety of future drone operations in urban airspace. First, reinforcement learning was used to define optimal heading range distributions with a layered airspace concept. Second, transition layers were reserved to facilitate the vertical deviation between cruising layers and conflict avoidance. The effects of traffic density, non-linear routes, and vertical deviation between layers were tested in an open-source airspace simulation platform. Results show that optimal structuring catered to the current traffic scenario improves airspace usage by correctly segmenting aircraft according to their flight routes. The number of conflicts and losses of minimum separation was reduced versus using a single, uniform airspace structure for all traffic scenarios, thus enabling higher airspace capacity.

Overview of Research of Machine Learning in Air Traffic Management

“…It has also been validated in extensive experiments in six regions of southwest Chinese airspace, demonstrating that the model can provide a more objective evaluation to help air traffic controllers understand airspace complexity. In more depth, Ribeiro et al used reinforcement learning to make the hierarchical design of the airspace more suitable for the flight paths and traffic conditions of aircraft in the current airspace, reducing the number of conflicts and loss of minimum separation, suggesting that this airspace structure is better able to distribute aircraft more evenly, thereby increasing airspace capacity [40].…”