This paper presents a new approach for analyzing the trajectory prediction performance of the FAA's Traffic Management Advisor (TMA). TMA is a deployed system that generates scheduled time-of-arrival constraints for en-route air traffic controllers in the US. The new automated analysis provides a repeatable evaluation of the current trajectory performance metrics, for new releases of TMA, in different traffic and airspace environments, and for current traffic situations. Using a wider set of data, it provides also a higher level of understanding on the causes of possible degradation of the trajectory prediction performance of TMA. The bulk of the work consisted of the development of the ability to filter flights not impacted by controller intervention. Identifying interrupted flights from recorded data is challenging but necessary for a fair and accurate performance test. Currently, no method for identification of flights exists other than a manual review of voice communications. The automated approach was tested with two data sets, from 2006 and 2013. The 2013 data consisted of 24 hours of traffic arriving into Dallas Forth Worth Airport. The results of the testing on this data set showed that the approach selected a statistically significant number of flights to validate the TMA trajectory predictor's performance against the system requirements. New metrics for the evaluation of the TMA trajectory predictor's performance are introduced and compared with the current set of metrics used by the FAA.

Many Air Navigation Service Providers (ANSPs) aim to report on the number of continuous descents conducted in their administered airspace. The method with which this reporting is undertaken varies between ANSPs, but often the metric includes measuring the distance flown at level altitude. While such a metric tests if a descent was continuous, it does not necessarily mean the descent was also efficient. In this paper a metric and deriving algorithm will be proposed that tests if a descent was a managed descent. The major difference between a managed descent and a continuous descent is that a managed descent is performed in a predictable manner to a pre-determined plan by the aircraft's Flight Management System (FMS). A managed descent is in general continuous, but not necessarily vice versa. Often, whenever it is referred to continuous descent, it is the predictability of an automation managed descent that the airspace operator ultimately wants to achieve, and what an ANSP should facilitate. Derivation of the proposed metric algorithm and its application is demonstrated based on flight data from Australia and the United States. While the first basic implementation of the metric has yet a number of limitations, its application identified differences between the efficiency of descent trajectories where the conventional metric, in terms of detecting level segments, did not. In addition, application on US flight data demonstrated how the metric can assist in analysing the performance of ground-based trajectory prediction tools.

No abstract

This paper presents a new methodology for validating an aircraft trajectory predictor, inspired by the lessons learned from a number of field trials, flight tests and simulation experiments for the development of trajectory-predictor-based automation. The methodology introduces new techniques and a new multi-staged approach to reduce the effort in identifying and resolving validation failures, avoiding the potentially large costs associated with failures during a single-stage, pass/fail approach. As a case study, the validation effort performed by the Federal Aviation Administration for its En Route Automation Modernization (ERAM) system is analyzed to illustrate the real-world applicability of this methodology. During this validation effort, ERAM initially failed to achieve six of its eight requirements associated with trajectory prediction and conflict probe. The ERAM validation issues have since been addressed, but to illustrate how the methodology could have benefited the FAA effort, additional techniques are presented that could have been used to resolve some of these issues. Using data from the ERAM validation effort, it is demonstrated that these new techniques could have identified trajectory prediction error sources that contributed to several of the unmet ERAM requirements.

No abstract