A Real-Time ATC Safety Monitoring Framework Using a Deep Learning Approach

Lin, Yi; Deng, Linjie; Chen, Zhengmao; Wu, Xinzhou; Zhang, Jianwei; Yang, Bo

doi:10.1109/tits.2019.2940992

Cited by 54 publications

(42 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ASR techniques have several potential applications in the ATC domain, including safety monitoring of live operations (4)(5)(6) and identification of anomalous aircraft trajectories (4). In another example of safety monitoring, Chen et al (5) developed a framework to automatically flag pilot-controller miscommunications (read back error detection) to prevent untoward incidents.…”

Section: Related Workmentioning

confidence: 99%

Automatic Speech Recognition for Air Traffic Control Communications

Badrinath

Balakrishnan

2021

Transportation Research Record: Journal of the Transportation R

View full text Add to dashboard Cite

A significant fraction of communications between air traffic controllers and pilots is through speech, via radio channels. Automatic transcription of air traffic control (ATC) communications has the potential to improve system safety, operational performance, and conformance monitoring, and to enhance air traffic controller training. We present an automatic speech recognition model tailored to the ATC domain that can transcribe ATC voice to text. The transcribed text is used to extract operational information such as call-sign and runway number. The models are based on recent improvements in machine learning techniques for speech recognition and natural language processing. We evaluate the performance of the model on diverse datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

Automatic Speech Recognition for Air Traffic Control Communications

Badrinath

Balakrishnan

2021

Transportation Research Record: Journal of the Transportation R

View full text Add to dashboard Cite

show abstract

“…When it comes to the ASR research in the ATC domain, our previous works have studied the independent system [1,11] and cascaded multilingual pipeline [2,12]. Airbus held a challenge that focused on translating the ATC speech and detecting the aircraft identification [10].…”

Section: Related Workmentioning

confidence: 99%

“…Although enormous efforts have been made to achieve the ASR task in the ATC domain, such as the independent system for a single language [1,10,11], the cascaded pipeline for multilingual recognition [2,12], it is also believed that an endto-end multilingual framework with the ability to address the above technical problems is the ultimate solution for the ASR research in the ATC domain. To this end, an improved end-toend ASR model is proposed to address the multilingual ASR task in this work, which is able to transcribe the speech signals into human-readable texts in Chinese characters and English words.…”

mentioning

confidence: 99%

Towards multilingual end‐to‐end speech recognition for air traffic control

Lin

Yang

Guo

et al. 2021

IET Intelligent Trans Sys

Self Cite

View full text Add to dashboard Cite

In this work, an end-to-end framework is proposed to achieve multilingual automatic speech recognition (ASR) in air traffic control (ATC) systems. Considering the standard ATC procedure, a recurrent neural network (RNN) based framework is selected to mine the temporal dependencies among speech frames. Facing the distributed feature space caused by the radio transmission, a hybrid feature embedding block is designed to extract high-level representations, in which multiple convolutional neural networks are designed to accommodate different frequency and temporal resolutions. The residual mechanism is performed on the RNN layers to improve the trainability and the convergence. To integrate the multilingual ASR into a single model and relieve the class imbalance, a special vocabulary is designed to unify the pronunciation of the vocabulary in Chinese and English, i.e., pronunciation-oriented vocabulary. The proposed model is optimized by the connectionist temporal classification loss and is validated on a real-world speech corpus (ATC-Speech). A character error rate of 4.4% and 5.9% is achieved for Chinese and English speech, respectively, which outperforms other popular approaches. Most importantly, the proposed approach achieves the multilingual ASR task in an end-to-end manner with considerable high performance. INTRODUCTIONAir traffic control (ATC) is an essential service provided by ground-based air traffic controllers (ATCOs) to guide the flight to be operated in a safe manner (i.e. prevent conflict), and further to organize and expedite the traffic flow. As the primary communication way between the ATCO and the aircrew, the spoken instruction through the very high frequency (VHF) radio transmission implies a wealth of contextualized situational information, which is important to the real-time ATC decisionmaking. In the current ATC management system, the ATC is a non-automatic procedure (human-in-the-loop) and is always regarded as a potential risk for the air traffic operation [1]. Numerous studies have demonstrated that monitoring the control conversation is a promising way to obtain real-time traffic dynamics [2,3,4], which benefits to formulate a closed-loop ATC management. To this end, the automatic speech recognition (ASR) technique, with the purpose of building the bridgeThis is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

“…Thus, the ASR system becomes a promising technique to bridge the speech communication and ATC system. Recently, more and more attention has been paid to employ the ASR techniques to empower ATC applications, such as the ATC assistance system [1], operational safety monitoring system [2,3], and the ATCO training system [4,5].…”

Section: Introductionmentioning

confidence: 99%

A Context-Aware Language Model to Improve the Speech Recognition in Air Traffic Control

et al. 2021

Self Cite

View full text Add to dashboard Cite

Recognizing isolated digits of the flight callsign is an important and challenging task for automatic speech recognition (ASR) in air traffic control (ATC). Fortunately, the flight callsign is a kind of prior ATC knowledge and is available from dynamic contextual information. In this work, we attempt to utilize this prior knowledge to improve the performance of the callsign identification by integrating it into the language model (LM). The proposed approach is named context-aware language model (CALM), which can be applied for both the ASR decoding and rescoring phase. The proposed model is implemented with an encoder–decoder architecture, in which an extra context encoder is proposed to consider the contextual information. A shared embedding layer is designed to capture the correlations between the ASR text and contextual information. The context attention is introduced to learn discriminative representations to support the decoder module. Finally, the proposed approach is validated with an end-to-end ASR model on a multilingual real-world corpus (ATCSpeech). Experimental results demonstrate that the proposed CALM outperforms other baselines for both the ASR and callsign identification task, and can be practically migrated to a real-time environment.

show abstract

A Real-Time ATC Safety Monitoring Framework Using a Deep Learning Approach

Cited by 54 publications

References 30 publications

Automatic Speech Recognition for Air Traffic Control Communications

Automatic Speech Recognition for Air Traffic Control Communications

Towards multilingual end‐to‐end speech recognition for air traffic control

A Context-Aware Language Model to Improve the Speech Recognition in Air Traffic Control

Contact Info

Product

Resources

About