Machine Learning of Air Traffic Controller Command Extraction Models for Speech Recognition Applications

Helmke, Hartmut; Kleinert, Matthias; Ohneiser, Oliver; Ehr, Heiko; Shetty, Shruthi

doi:10.1109/dasc50938.2020.9256484

Cited by 14 publications

(14 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The WERs for all datasets, particularly ANS CR and ACG are much higher as compared to what was reported in(Helmke et al 2020) because of some updates made to the transcription rules for callsign designators. The gold transcriptions were updated accordingly, but the automatic transcriptions were not modified in order to show the potential of the callsign extraction algorithm also on noise data.…”

mentioning

confidence: 61%

Early Callsign Highlighting using Automatic Speech Recognition to Reduce Air Traffic Controller Workload

Shetty¹,

Helmke²,

Kleinert³

et al. 2022

Human Factors in Transportation

View full text Add to dashboard Cite

The primary task of an air traffic controller (ATCo) is to issue instructions to pi-lots. However, the first verbal communication contact is often initiated by the pi-lot. Hence, the ATCo needs to search for the aircraft radar label that corresponds to the callsign uttered by the pilot. Therefore, it would be useful to have a control-ler assistance system, which recognizes and highlights the spoken callsign in the ATCo display as early as possible, directly from the speech data. Therefore, we propose to use an automatic speech recognition (ASR) system to first obtain the speech-to-text transcription, followed by extracting the spoken callsign from the transcription. As a high performance in callsign recognition is required, we use surveillance data, which significantly reduces callsign recognition error rates. When using ASR transcriptions for ATCo utterances of Isavia data (HAAWAII project ), we initially obtain a callsign recognition error rate of 6.2%, which im-proves to 2.8% when surveillance data information is used.For the ATC operational speech data obtained from NATS air navigation service provider for London approach area, currently we obtain a callsign recognition rate of 93.8% for both ATCo and pilot utterances on automatic transcriptions which are generated by an ASR system with a word error rate of 5.1%. However, when surveillance data is not used, the callsign recognition rate drops significantly to 82.7%, indicating the importance of using surveillance data while recognizing callsigns. Once the callsign is spoken, we are able to recognize it within a second, which would be of great value to ATCos especially in situations of high traffic constituting high workload.

show abstract

mentioning

confidence: 61%

Early Callsign Highlighting using Automatic Speech Recognition to Reduce Air Traffic Controller Workload

Shetty¹,

Helmke²,

Kleinert³

et al. 2022

Human Factors in Transportation

View full text Add to dashboard Cite

show abstract

“…Authors in [91] show that in every hundredth ATC communication, an error may occur, and in [92], the authors show that the error may occur in every sixteenth communication. The possibility to detect such error still remains a challenge, as shown in this recent work [93]. Although, in general, read-back errors are quite rare, preventing even one incident due to automatic RBED can make an important difference in ensuring ATM safety.…”

Section: Read-back Error Detectionmentioning

confidence: 93%

Lessons Learned in Transcribing 5000 h of Air Traffic Control Communications for Robust Automatic Speech Understanding

Zuluaga-Gomez,

Nigmatulina,

Prasad

et al. 2023

Aerospace

View full text Add to dashboard Cite

Voice communication between air traffic controllers (ATCos) and pilots is critical for ensuring safe and efficient air traffic control (ATC). The handling of these voice communications requires high levels of awareness from ATCos and can be tedious and error-prone. Recent attempts aim at integrating artificial intelligence (AI) into ATC communications in order to lessen ATCos’s workload. However, the development of data-driven AI systems for understanding of spoken ATC communications demands large-scale annotated datasets, which are currently lacking in the field. This paper explores the lessons learned from the ATCO2 project, which aimed to develop an unique platform to collect, preprocess, and transcribe large amounts of ATC audio data from airspace in real time. This paper reviews (i) robust automatic speech recognition (ASR), (ii) natural language processing, (iii) English language identification, and (iv) contextual ASR biasing with surveillance data. The pipeline developed during the ATCO2 project, along with the open-sourcing of its data, encourages research in the ATC field, while the full corpus can be purchased through ELDA. ATCO2 corpora is suitable for developing ASR systems when little or near to no ATC audio transcribed data are available. For instance, the proposed ASR system trained with ATCO2 reaches as low as 17.9% WER on public ATC datasets which is 6.6% absolute WER better than with “out-of-domain” but gold transcriptions. Finally, the release of 5000 h of ASR transcribed speech—covering more than 10 airports worldwide—is a step forward towards more robust automatic speech understanding systems for ATC communications.

show abstract

“…10, it can be seen that the CER metric has apparent limitations when measuring the performance of an ATC speech recognition system from the perspective of the effectiveness of control instructions. (2) The DCNN model has the worst generalization ability. In Table 10, the CER of DCNN is 7.5% in the test dataset with the same distribution as the training data.…”

Section: Robustness Verification Of Modelsmentioning

confidence: 99%

“…Therefore, ensuring the effective transmission of ATC instruction information is crucial for air traffic safety. In ATC, control instructions are primarily conveyed through pilot-controller voice communications (PCVCs) [1,2] indicates that digital automation is critical in addressing challenges in air traffic flow management (ATFM), such as safety, capacity, efficiency, and environmental aspects, and future Controller-Pilot Data Link Communication (CPDLC) will replace PCVCs. Currently, the Civil Aviation Administration of China (CAAC) is promoting intelligent air traffic management (ATM), which includes key technologies such as multimodal fusion, ASR, ATC instructions intent recognition, automatic response to ATC instructions, and semantic verification of ATC instructions [3].…”

Section: Introductionmentioning

confidence: 99%

A Robust Conformer-Based Speech Recognition Model for Mandarin Air Traffic Control

Jiang,

Pan,

Zhang

et al. 2023

CMC

View full text Add to dashboard Cite

This study aims to address the deviation in downstream tasks caused by inaccurate recognition results when applying Automatic Speech Recognition (ASR) technology in the Air Traffic Control (ATC) field. This paper presents a novel cascaded model architecture, namely Conformer-CTC/Attention-T5 (CCAT), to build a highly accurate and robust ATC speech recognition model. To tackle the challenges posed by noise and fast speech rate in ATC, the Conformer model is employed to extract robust and discriminative speech representations from raw waveforms. On the decoding side, the Attention mechanism is integrated to facilitate precise alignment between input features and output characters. The Text-To-Text Transfer Transformer (T5) language model is also introduced to handle particular pronunciations and code-mixing issues, providing more accurate and concise textual output for downstream tasks. To enhance the model's robustness, transfer learning and data augmentation techniques are utilized in the training strategy. The model's performance is optimized by performing hyperparameter tunings, such as adjusting the number of attention heads, encoder layers, and the weights of the loss function. The experimental results demonstrate the significant contributions of data augmentation, hyperparameter tuning, and error correction models to the overall model performance. On the Our ATC Corpus dataset, the proposed model achieves a Character Error Rate (CER) of 3.44%, representing a 3.64% improvement compared to the baseline model. Moreover, the effectiveness of the proposed model is validated on two publicly available datasets. On the AISHELL-1 dataset, the CCAT model achieves a CER of 3.42%, showcasing a 1.23% improvement over the baseline model. Similarly, on the LibriSpeech dataset, the CCAT model achieves a Word Error Rate (WER) of 5.27%, demonstrating a performance improvement of 7.67% compared to the baseline model. Additionally, this paper proposes an evaluation criterion for assessing the robustness of ATC speech recognition systems. In robustness evaluation experiments based on this criterion, the proposed model demonstrates a performance improvement of 22% compared to the baseline model.

show abstract

Machine Learning of Air Traffic Controller Command Extraction Models for Speech Recognition Applications

Cited by 14 publications

References 9 publications

Early Callsign Highlighting using Automatic Speech Recognition to Reduce Air Traffic Controller Workload

Early Callsign Highlighting using Automatic Speech Recognition to Reduce Air Traffic Controller Workload

Lessons Learned in Transcribing 5000 h of Air Traffic Control Communications for Robust Automatic Speech Understanding

A Robust Conformer-Based Speech Recognition Model for Mandarin Air Traffic Control

Contact Info

Product

Resources

About