Learning Assurance Analysis for Further Certification Process of Machine Learning Techniques: Case-Study Air Traffic Conflict Detection Predictor

Abstract

Designing and developing artificial intelligence (AI)-based systems that can be trusted justifiably is one of the main issues aviation must face in the coming years. European Union Aviation Safety Agency (EASA) has developed a user guide that could be potentially transformed as means of compliance for future AI-based regulation. Designers and developers must understand how the learning assurance process of any machine learning (ML) model impacts trust. ML is a narrow branch of AI that uses statistical models to perform predictions. This work deals with the learning assurance process for ML-based systems in the field of air traffic control. A conflict detection tool has been developed to identify separation infringements among aircraft pairs, and the ML algorithm used for classification and regression was extreme gradient boosting. This paper analyses the validity and adaptability of EASA W-shaped methodology for ML-based systems. The results have identified the lack of the EASA W-shaped methodology in time-dependent analysis, by showing how time can impact ML algorithms designed in the case where no time requirements are considered. Another meaningful conclusion is, for systems that depend highly on when the prediction is made, classification and regression metrics cannot be one-size-fits-all because they vary over time.

Artificial Intelligence in Aviation: New Professionals for New Technologies

“…The methodology for Machine Learning (ML) systems in air traffic control, specifically in a conflict detection tool, is evaluated in [10]. Results revealed that the methodology lacks time-dependent analysis, indicating that for certain systems, standard classification and regression metrics cannot be universally applied due to their variability over time.…”

Machine Learning-based Aircraft Conflict Prediction: A Systematic Literature Review

“…A conflict prediction tool is presented in [14] which detect SMI between pairs of aircraft. This data-driven approach proposed in this study extracts aircraft trajectory data from the OpenSky Network which provides real world ADS-B flight trajectories.…”