The aim of this work is to investigate the possibility of using machine learning (ML) methods in order to generate novel, safety-relevant knowledge from existing flight data. Airlines routinely generate vast amounts of flight data from routine monitoring, but the concept of extracting safety knowledge from this data is still based on detecting exceedances of expert-defined thresholds. This system is conceptually unable to detect novel occurrences for which no such filters exist. ML techniques are able to close this gap. This paper first reviews the literature to select an appropriate ML method. A form of unsupervised learning called "Local Outlier Probability" is selected. Next, an appropriate feature space is developed and implemented in the flight data monitoring system of a supporting airline to generate the dataset. This dataset is cleaned and the outlier calculation performed. The results are statistically analysed. Furthermore, the top outliers are reviewed by the airline's review pilots in the same way as the traditional exceedance events. Last, the severities and safety relevance of both types of events are compared. This work successfully shows that the chosen approach is able to reduce the number of undetected safety-relevant occurrences by finding novel occurrence types which were undetected by a contemporary and mature flight data monitoring system. This research builds on recent literature by developing a novel method which can be scaled to work in an airline production environment with large datasets, as demonstrated by the efficient analysis of 1.2 million flights.

No abstract

Safety occurrence reports can contain valuable information on how incidents occur, revealing knowledge that can assist safety practitioners. This paper presents and discusses a literature review exploring how Natural Language Processing (NLP) has been applied to occurrence reports within safety-critical industries, informing further research on the topic and highlighting common challenges. Some of the uses of NLP include the ability for occurrence reports to be automatically classified against categories, and entities such as causes and consequences to be extracted from the text as well as the semantic searching of occurrence databases. The review revealed that machine learning models form the dominant method when applying NLP, although rule-based algorithms still provide a viable option for some entity extraction tasks. Recent advances in deep learning models such as Bidirectional Transformers for Language Understanding are now achieving a high accuracy while eliminating the need to substantially pre-process text. The construction of safety-themed datasets would be of benefit for the application of NLP to occurrence reporting, as this would allow the fine-tuning of current language models to safety tasks. An interesting approach is the use of topic modelling, which represents a shift away from the prescriptive classification taxonomies, splitting data into “topics”. Where many papers focus on the computational accuracy of models, they would also benefit from real-world trials to further inform usefulness. It is anticipated that NLP will soon become a mainstream tool used by safety practitioners to efficiently process and gain knowledge from safety-related text.

A substantial amount of effort and resource is applied to the design of aircraft systems to reduce risk to life and improve safety. This is often applied through a variety of safety assessment methods, one of which being Hazard and Operability (HAZOP) Studies. Once an air system is in-service, it is common for flight data to be collected and analysed to validate the original safety assessment. However, the operator of the air system generates and stores a substantial amount of safety knowledge within free-text occurrence reports. These allow maintainers and aircrew to report occurrences, often describing hazards and associated detail revealing consequences and causes. A lack of resource means it is difficult for safety professionals to manually review these occurrences and although occurrences are classified against a set taxonomy (e.g., birdstrike, technical failure) this lacks the granularity to apply to a specific safety analysis. To resolve this, the paper presents the development of a novel Natural Language Processing (NLP) framework for extracting causes, consequences, and hazards from free-text occurrence reports in order to validate and inform an aircraft sub-system HAZOP study. Specifically using a combination of rule-based phrase matching with a spaCy Named Entity Recognition (NER) model. It is suggested that the framework could form a continual improvement process whereby the findings drive updates to the HAZOP, in turn updating the rules and model, therefore improving accuracy and hazard identification over time.

One area the aviation industry is grappling with is the quantification of the probability of occurrence of safety incidents. Currently, aviation professionals involved in safety risk management mostly rely on collective experience to determine probability of incident occurrences and apply it to the International Civil Aviation Organisation (ICAO) matrix or equivalent to evaluate the risk. A number of limitations linked to the use of risk matrices will be explored in this paper. It is the aim of this paper to explore statistical methods that can be used to determine the probability of safety occurrences and come up with an algorithm that can be used by airlines using available safety data. The novelty of this research is that it combines the exploration of use of statistical techniques to quantitatively assess risk using Flight Data Monitoring (FDM) and other data, with acceptability of Safety Risk Management (SRM) data analytics by operational personnel. The paper also explores the contributory factors leading to the reluctance of operational personnel to use data analytics to inform their risk assessments despite the increasing availability of operational data and advancement in technology.