A Two-Step Approach to Leverage Contextual Data: Speech Recognition in Air-Traffic Communications

Nigmatulina, Iuliia; Zuluaga-Gómez, Juan; Prasad, Amrutha; Sarfjoo, Seyyed Saeed; Motlíček, Petr

doi:10.1109/icassp43922.2022.9746563

Cited by 12 publications

(17 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Thousands are correctly identified (100%); • Hundreds are correctly identified (100%); • Numbers between 11 and 99 (e.g., 13,18,34) have very high recognition rates (98%);…”

Section: Technical Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Flight Callsign Identification on a Controller Working Position: Real-Time Simulation and Analysis of Operational Recordings

et al. 2023

View full text Add to dashboard Cite

In the air traffic management (ATM) environment, air traffic controllers (ATCos) and flight crews, (FCs) communicate via voice to exchange different types of data such as commands, readbacks (confirmation of reception of the command) and information related to the air traffic environment. Speech recognition can be used in these voice exchanges to support ATCos in their work; each time a flight identification or callsign is mentioned by the controller or the pilot, the flight is recognised through automatic speech recognition (ASR) and the callsign is highlighted on the ATCo screen to increase their situational awareness and safety. This paper presents the work that is being performed within SESAR2020-founded solution PJ.10-W2-96 ASR in callsign recognition via voice by Enaire, Indra, and Crida using ASR models developed jointly by EML Speech Technology GmbH (EML) and Crida. The paper describes the ATCo speech environment and presents the main requirements impacting the design, the implementation performed, and the outcomes obtained using real operation communications and real-time simulations. The findings indicate a way forward incorporating partial recognition of callsigns and enriching the phonetization of company names to improve the recognition rates, currently set at 84–87% for controllers and 49–67% for flight crew.

show abstract

“…• Thousands are correctly identified (100%); • Hundreds are correctly identified (100%); • Numbers between 11 and 99 (e.g., 13,18,34) have very high recognition rates (98%);…”

Section: Technical Resultsmentioning

confidence: 99%

“…Information extraction from written text can follow very different approaches and several factors (language, domain, entity type) impact the selected technique [11,12]. The extraction of information in the ATM domain has mainly used knowledge-based methods and machine learning models [5,13].…”

mentioning

confidence: 99%

Automatic Flight Callsign Identification on a Controller Working Position: Real-Time Simulation and Analysis of Operational Recordings

et al. 2023

View full text Add to dashboard Cite

show abstract

“…In [13] contextual information (also known as contextual biasing) via n-grams composition (in the HCLG graph 7 ) is merged with semi-supervised learning techniques to further decrease word error rates (WER) on an ASR designed for ATC. Boosting of contextual knowledge during and after decoding has also been explored in [29,30,31,32], where a set of target n-grams are added to further decrease WERs.…”

Section: Related Workmentioning

confidence: 99%

“…LiveATC-Test: the test set is gathered from LiveATC 10 data recorded from publicly accessible VHF radio channels, as a part of ATCO2 project [13,32], and includes pilot and ATCO recordings with accented English from airports located in U.S., Czech Republic, Ireland, Netherlands, and Switzerland. We consider LiveATC-Test as low quality speech data set i.e., signal-to-noise (SNR) ratios goes from 5 to 15 dB [22].…”

Section: Private Databasesmentioning

confidence: 99%

How Does Pre-Trained Wav2Vec 2.0 Perform on Domain-Shifted Asr? an Extensive Benchmark on Air Traffic Control Communications

Zuluaga-Gómez

Prasad

Nigmatulina

et al. 2023

2022 IEEE Spoken Language Technology Workshop (SLT)

Self Cite

View full text Add to dashboard Cite

Recent work on self-supervised pre-training focus on leveraging large-scale unlabeled speech data to build robust end-to-end (E2E) acoustic models (AM) that can be later fine-tuned on downstream tasks e.g., automatic speech recognition (ASR). Yet, few works investigated the impact on performance when the data properties substantially differ between the pre-training and fine-tuning phases, termed domain shift. We target this scenario by analyzing the robustness of Wav2Vec 2.0 and XLS-R models on downstream ASR for a completely unseen domain, air traffic control (ATC) communications. We benchmark these two models on several open-source and challenging ATC databases with signal-to-noise ratio between 5 to 20 dB. Relative word error rate (WER) reductions between 20% to 40% are obtained in comparison to hybrid-based ASR baselines by only fine-tuning E2E acoustic models with a smaller fraction of labeled data. We analyze WERs on the low-resource scenario and gender bias carried by one ATC dataset.

show abstract

“…A semi-supervised learning approach for enhancing ASR in the ATM domain was employed in [11,12,14]. In [15][16][17], the authors aimed to improve the recognition of the callsigns in ASR by integrating surveillance data. Finally, the authors of [18] investigated the effect of fine-tuning large pre-trained models, trained using a Transformer architecture, for application in the ATC domain.…”

Section: Automatic Speech Recognition (Asr)mentioning

confidence: 99%

An Automatic Speaker Clustering Pipeline for the Air Traffic Communication Domain

Khalil,

Prasad,

Motlicek

et al. 2023

Aerospace

View full text Add to dashboard Cite

In air traffic management (ATM), voice communications are critical for ensuring the safe and efficient operation of aircraft. The pertinent voice communications—air traffic controller (ATCo) and pilot—are usually transmitted in a single channel, which poses a challenge when developing automatic systems for air traffic management. Speaker clustering is one of the challenges when applying speech processing algorithms to identify and group the same speaker among different speakers. We propose a pipeline that deploys (i) speech activity detection (SAD) to identify speech segments, (ii) an automatic speech recognition system to generate the text for audio segments, (iii) text-based speaker role classification to detect the role of the speaker—ATCo or pilot in our case—and (iv) unsupervised speaker clustering to create a cluster of each individual pilot speaker from the obtained speech utterances. The speech segments obtained by SAD are input into an automatic speech recognition (ASR) engine to generate the automatic English transcripts. The speaker role classification system takes the transcript as input and uses it to determine whether the speech was from the ATCo or the pilot. As the main goal of this project is to group the speakers in pilot communication, only pilot data acquired from the classification system is employed. We present a method for separating the speech parts of pilots into different clusters based on the speaker’s voice using agglomerative hierarchical clustering (AHC). The performance of the speaker role classification and speaker clustering is evaluated on two publicly available datasets: the ATCO2 corpus and the Linguistic Data Consortium Air Traffic Control Corpus (LDC-ATCC). Since the pilots’ real identities are unknown, the ground truth is generated based on logical hypotheses regarding the creation of each dataset, timing information, and the information extracted from associated callsigns. In the case of speaker clustering, the proposed algorithm achieves an accuracy of 70% on the LDC-ATCC dataset and 50% on the more noisy ATCO2 dataset.

show abstract

A Two-Step Approach to Leverage Contextual Data: Speech Recognition in Air-Traffic Communications

Cited by 12 publications

References 17 publications

Automatic Flight Callsign Identification on a Controller Working Position: Real-Time Simulation and Analysis of Operational Recordings

Automatic Flight Callsign Identification on a Controller Working Position: Real-Time Simulation and Analysis of Operational Recordings

How Does Pre-Trained Wav2Vec 2.0 Perform on Domain-Shifted Asr? an Extensive Benchmark on Air Traffic Control Communications

An Automatic Speaker Clustering Pipeline for the Air Traffic Communication Domain

Contact Info

Product

Resources

About