Automatic Flight Callsign Identification on a Controller Working Position: Real-Time Simulation and Analysis of Operational Recordings

Abstract

In the air traffic management (ATM) environment, air traffic controllers (ATCos) and flight crews, (FCs) communicate via voice to exchange different types of data such as commands, readbacks (confirmation of reception of the command) and information related to the air traffic environment. Speech recognition can be used in these voice exchanges to support ATCos in their work; each time a flight identification or callsign is mentioned by the controller or the pilot, the flight is recognised through automatic speech recognition (ASR) and the callsign is highlighted on the ATCo screen to increase their situational awareness and safety. This paper presents the work that is being performed within SESAR2020-founded solution PJ.10-W2-96 ASR in callsign recognition via voice by Enaire, Indra, and Crida using ASR models developed jointly by EML Speech Technology GmbH (EML) and Crida. The paper describes the ATCo speech environment and presents the main requirements impacting the design, the implementation performed, and the outcomes obtained using real operation communications and real-time simulations. The findings indicate a way forward incorporating partial recognition of callsigns and enriching the phonetization of company names to improve the recognition rates, currently set at 84–87% for controllers and 49–67% for flight crew.

Ensuring Safety for Artificial-Intelligence-Based Automatic Speech Recognition in Air Traffic Control Environment

“…ENAIRE, Indra and CRIDA, conducted an exercise to validate the performance of the pre-industrial ASR prototype covering the following use cases [4]: Sequence diagrams were produced and analyzed for each use case. The sequence diagrams were used for the walkthrough with subject matter experts to identify potential hazards, meaning each situation that could trigger the unsafe situation.…”

“…ENAIRE, Indra and CRIDA, conducted an exercise to validate the performance of the pre-industrial ASR prototype covering the following use cases [4]: This validation exercise used two complementary approaches aiming at providing evidence of ASR applications' performance by providing the following outputs:…”

“…This is due in part to the fact that European regulation [2] requires any new technology introduced into the operational environment undergoes a rigorous safety assessment conducted at the design phase, providing relevant evidence that the physical design satisfies the design requirements. The safety of two ASR prototypes for air traffic control (ATC) purposes supported by artificial intelligence (AI) has been assessed by Integra in accordance with the Single European Sky Air Traffic Management Research (SESAR) safety reference materials [3] in the course of SESAR2020 project PJ.10-W2-96-ASR [4,5]. The safety assessment includes several steps, starting at the design level with the assessment of the introduction of a new system-or a change to an existing system-for the identification of any hazards introduced by the new system elements, and possible associated increase in risks.…”

“…The basis of this paper, i.e., the follow-up project PJ.10-W2-96 [4,5] continued developing the application with the aim of demonstrating the technology feasibility in relevant operational environment (TRL6). In parallel, the project PJ.05-W2-97 HMI Interaction modes for Airport Tower aimed to develop an ASR system for an aerodrome control tower environment [8].…”

“…An additional safety net for AI applications in ASR is intended to ensure that errors in AI-based speech recognition do not have any negative effects on the overall system [16]. The benefits of callsign recognition from flight crew utterances and highlighting the callsign at an en-route CWP HMI were investigated in [4]. The study demonstrated feasibility of the integrated ASR system for the identification of callsigns from flight crew utterances which provide benefits in terms of workload and situational awareness.…”