A computer vision framework to detect, track and identify objects in a wide field-of-view airport-airside environment

Abstract

To accommodate the steady growth of air traffic, airports worldwide need to increase their capacity by expanding infrastructure and optimizing air traffic procedures. However, as airports add new infrastructure (runways, taxiways, and aprons) into operation, they also increase the challenge of airport airside management. This may lead to an increase in controller workload as well as runway and taxiway incursion events. This problem is compounded by unauthorized access to the control zone by unauthorized and non-cooperative objects, such as drones. Air traffic controllers now are therefore required to monitor more traffic and direct complex operations. For effective airport airside management, controllers must detect aircraft movement on the ground. They also must coordinate push-back procedures, taxiway routing, and runway sequencing for effective airside traffic circulation. Controllers are traditionally located in a tall control tower to manage air traffic through an out-of-window view. Although physical towers have served airside control well, they have several limitations. First, a physical tower is costly for a small airport because the fixed costs for providing air traffic service are independent of the traffic. Secondly, one physical tower might not be sufficient to visually cover a large airport. For full visual traffic coverage of a large airport, the geometric height of a tower can be sustainably high which poses an obstruction to navigation in the airport control zone.To enhance safety and improve operational efficiency, the concept of the digital tower has been developed. A digital tower replaces the out-of-window view of a conventional tower ix with a visualization system provided by a network of high-resolution cameras. By using such a visualization system, an airport can have more than one digital tower if necessary, resulting in greater visual coverage. The visibility can be further enhanced by infrared and pan-tiltzoom cameras. With a digital system, digital towers can provide many enhanced functions to assist controllers. However, the airport airside is a wide field-of-view environment with dynamic traffic and complex operations, and existing computer vision systems at airports only focus on a portion of the airport airside, such as detecting an aircraft on the runway, tracking moving aircraft, or detecting objects on an apron. Moreover, such systems do not consider gate site management, such as push-back control or turnaround monitoring.The objective of this thesis is to develop a computer vision framework to detect, track and identify aircraft and relevant objects (cooperative and non-cooperative) in a wide field-of-view (multiple cameras) airside for effective airside monitoring and turnaround management. The framework designs a specific Convolutional Neural Network, namely AirNet, customized for the unique characteristics of the airport airside environment. By using depthwise convolution operation, the computational costs are significantly reduced while maintaining high detection performance...