Emergency Vehicles Audio Detection and Localization in Autonomous Driving

Abstract

Emergency vehicles in service have right-of-way over all other vehicles. Hence, all other vehicles are supposed to take proper actions to yield emergency vehicles with active sirens. As this task requires the cooperation between ears and eyes for human drivers, it also needs audio detection as a supplement to vision-based algorithms for fully autonomous driving vehicles. In urban driving scenarios, we need to know both the existence of emergency vehicles and their relative positions to us to decide the proper actions. We present a novel system from collecting the real-world siren data to the deployment of models using only two cost-efficient microphones. We are able to achieve promising performance for each task separately, especially within the crucial 10m to 50m distance range to react (the size of our ego vehicle is around 5m in length and 2m in width). The recall rate to determine the existence of sirens is 99.16% , the median and mean angle absolute error is 9.64°and 19.18°respectively, and the median and mean distance absolute error of 9.30m and 10.58m respectively within that range. We also benchmark various machine learning approaches that can determine the siren existence and sound source localization which includes direction and distance simultaneously within 50ms of latency.

Real-Time Acoustic Perception for Automotive Applications

“…Despite the wide literature on acoustic scene analysis, works targeting automotive applications are still limited. Some authors have tackled the detection and localization of emergency sound events, such as car horns and emergency sirens in an urban scenario [13]- [19]. These works can be split into the ones addressing only the detection problem [13], [14], [16], [17] and the ones targeting localization as well [15], [18], [19].…”

“…Some authors have tackled the detection and localization of emergency sound events, such as car horns and emergency sirens in an urban scenario [13]- [19]. These works can be split into the ones addressing only the detection problem [13], [14], [16], [17] and the ones targeting localization as well [15], [18], [19]. The proposed solutions to both problems are mostly based on endto-end machine learning and deep learning methods, that have proven to provide an increased robustness to strong background noise and complex dynamic acoustic scenes as compared to traditional signal processing techniques [15].…”

“…First, a featureextraction step takes as input the audio signal recorded by one (for the detection-only problem) or multiple (if the localization is addressed too) microphones and builds a corresponding representation to be used as input to a neural network. Most methods use time-frequency feature representations, such as spectrograms [13], [14], [16], [19], gammatonegrams [14], [15], Mel-frequency cepstral coefficients (MFCCs) [13], [17], [19], or, less commonly, gammatone-frequency cesptral coefficients (GFCCs), constant-Q transform (CQT) and chromagrams [17]. Others take the raw waveform of the windowed audio signal as input feature [18].…”

Few-Shot Emergency Siren Detection

“…Besides the neural architectures, the regularization and optimization techniques, and the performance achieved, most of the ESD works have in common a large amount of annotated audio data for training [10][11][12]. Typically, a large dataset represents the requirement of modern supervised deep learning models to build a robust system for detecting and classifying sound events [13][14][15], especially rare ones [16,17].…”

Multi-ACCDOA: Localizing and Detecting Overlapping Sounds from the Same Class with Auxiliary Duplicating Permutation Invariant Training

“…Sound event localization and detection (SELD) involves identifying the direction-of-arrival (DOA) and the type of sound events. SELD has played an essential role in many applications, such as surveillance [1,2], bio-diversity monitoring [3], and context-aware devices [4,5]. Recent competitions such as the DCASE challenge show significant progress in the SELD research area using neuralnetwork (NN)-based methods [6].…”