Automated Interpretation of Air Traffic Control Communication: The Journey from Spoken Words to a Deeper Understanding of the Meaning

Kleinert, Matthias; Helmke, Hartmut; Shetty, Shruthi; Ohneiser, Oliver; Ehr, Heiko; Prasad, Amrutha; Motlíček, Petr; Harfmann, Julia

doi:10.1109/dasc52595.2021.9594387

Cited by 8 publications

(5 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A callsign is said to be rejected if NO_CALLSIGN is extracted, but a valid callsign exists in the gold annotation (also referred to as deletions). Callsign rejection rate is the percentage of gold callsigns which are not extracted, i.e., the percentage of NO_CALLSIGN extractions (Kleinert et al 2021). The metrics are illustrated in Table 3 and the complete table with an example can be found in (Kleinert et al 2021).…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Early Callsign Highlighting using Automatic Speech Recognition to Reduce Air Traffic Controller Workload

Shetty¹,

Helmke²,

Kleinert³

et al. 2022

Human Factors in Transportation

View full text Add to dashboard Cite

The primary task of an air traffic controller (ATCo) is to issue instructions to pi-lots. However, the first verbal communication contact is often initiated by the pi-lot. Hence, the ATCo needs to search for the aircraft radar label that corresponds to the callsign uttered by the pilot. Therefore, it would be useful to have a control-ler assistance system, which recognizes and highlights the spoken callsign in the ATCo display as early as possible, directly from the speech data. Therefore, we propose to use an automatic speech recognition (ASR) system to first obtain the speech-to-text transcription, followed by extracting the spoken callsign from the transcription. As a high performance in callsign recognition is required, we use surveillance data, which significantly reduces callsign recognition error rates. When using ASR transcriptions for ATCo utterances of Isavia data (HAAWAII project ), we initially obtain a callsign recognition error rate of 6.2%, which im-proves to 2.8% when surveillance data information is used.For the ATC operational speech data obtained from NATS air navigation service provider for London approach area, currently we obtain a callsign recognition rate of 93.8% for both ATCo and pilot utterances on automatic transcriptions which are generated by an ASR system with a word error rate of 5.1%. However, when surveillance data is not used, the callsign recognition rate drops significantly to 82.7%, indicating the importance of using surveillance data while recognizing callsigns. Once the callsign is spoken, we are able to recognize it within a second, which would be of great value to ATCos especially in situations of high traffic constituting high workload.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Callsign rejection rate is the percentage of gold callsigns which are not extracted, i.e., the percentage of NO_CALLSIGN extractions (Kleinert et al 2021). The metrics are illustrated in Table 3 and the complete table with an example can be found in (Kleinert et al 2021).…”

Section: Methodsmentioning

confidence: 99%

Early Callsign Highlighting using Automatic Speech Recognition to Reduce Air Traffic Controller Workload

Shetty¹,

Helmke²,

Kleinert³

et al. 2022

Human Factors in Transportation

View full text Add to dashboard Cite

show abstract

“…Concept Recognition: Each time a recognized word sequence is forwarded, it is analyzed by the Concept Recognition module. The analysis result is then transformed into relevant ATC concepts as defined by SESAR project PJ.16-04 CWP HMI [2] and extended by the HAAWAII project [30], as shown in Figure 8. A command at the semantic level consists of a type, values, unit, qualifier, etc., as shown in Figure 8.…”

Section: Voice Activity Detection (Vad)mentioning

confidence: 99%

Validating Automatic Speech Recognition and Understanding for Pre-Filling Radar Labels—Increasing Safety While Reducing Air Traffic Controllers’ Workload

Ahrenhold,

Helmke,

Mühlhausen

et al. 2023

Aerospace

Self Cite

View full text Add to dashboard Cite

Automatic speech recognition and understanding (ASRU) for air traffic control (ATC) has been investigated in different ATC environments and applications. The objective of this study was to quantify the effect of ASRU support for air traffic controllers (ATCos) radar label maintenance in terms of safety and human performance. Therefore, an implemented ASRU system was validated within a human-in-the-loop environment by ATCos in different traffic-density scenarios. In the baseline condition, ATCos performed radar label maintenance by entering verbally instructed ATC commands with a mouse and keyboard. In the proposed solution, ATCos were supported by ASRU, which achieved a command recognition rate of 92.5% with a command error rate of 2.4%. ASRU support reduced the number of wrong or missing inputs from ATCos into the radar label by a factor of two, which contemporaneously improved their situational awareness. Furthermore, ATCos where able to perform more successful secondary tasks when using ASRU support, indicating a greater capacity to handle unexpected events. The results from NASA TLX showed that the perceived workload decreased with a statistical significance of 4.3% across all scenarios. In conclusion, this study provides evidence that using ASRU for radar label maintenance can significantly reduce workload and improve flight safety.

show abstract

“…The speech understanding implementation is based on a rule-based algorithm that identifies the relevant parts step by step and converts them into ATC commands. For more information, see [39].…”

Section: Speech Understandingmentioning

confidence: 99%

Safety Aspects of Supporting Apron Controllers with Automatic Speech Recognition and Understanding Integrated into an Advanced Surface Movement Guidance and Control System

et al. 2023

Self Cite

View full text Add to dashboard Cite

The information air traffic controllers (ATCos) communicate via radio telephony is valuable for digital assistants to provide additional safety. Yet, ATCos have to enter this information manually. Assistant-based speech recognition (ABSR) has proven to be a lightweight technology that automatically extracts and successfully feeds the content of ATC communication into digital systems without additional human effort. This article explains how ABSR can be integrated into an advanced surface movement guidance and control system (A-SMGCS). The described validations were performed in the complex apron simulation training environment of Frankfurt Airport with 14 apron controllers in a human-in-the-loop simulation in summer 2022. The integration significantly reduces the workload of controllers and increases safety as well as overall performance. Based on a word error rate of 3.1%, the command recognition rate was 91.8% with a callsign recognition rate of 97.4%. This performance was enabled by the integration of A-SMGCS and ABSR: the command recognition rate improves by more than 15% absolute by considering A-SMGCS data in ABSR.

show abstract

Automated Interpretation of Air Traffic Control Communication: The Journey from Spoken Words to a Deeper Understanding of the Meaning

Cited by 8 publications

References 4 publications

Early Callsign Highlighting using Automatic Speech Recognition to Reduce Air Traffic Controller Workload

Early Callsign Highlighting using Automatic Speech Recognition to Reduce Air Traffic Controller Workload

Validating Automatic Speech Recognition and Understanding for Pre-Filling Radar Labels—Increasing Safety While Reducing Air Traffic Controllers’ Workload

Safety Aspects of Supporting Apron Controllers with Automatic Speech Recognition and Understanding Integrated into an Advanced Surface Movement Guidance and Control System

Contact Info

Product

Resources

About