Sound Source Localization Using Deep Learning Models

Yalta, Nelson; Nakadai, Kazuhiro; Ogata, Tetsuya

doi:10.20965/jrm.2017.p0037

Cited by 112 publications

(83 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…cross-correlation [1,5,8], inter-channel phase and level difference [9,10]) or the subspace-based features [2,11,12], whereas SNS classification normally requires features computed from the power spectrum [13,14]. Recently, it has been shown that instead of applying complicated feature extraction, we can directly use the power spectrum as the inputs for neural networkbased sound source localization [15]. However, unlike in [15], our method employs the real and imaginary parts of the STFT, preserving both the power and phase information.…”

Section: Network Inputmentioning

confidence: 99%

“…Recently, it has been shown that instead of applying complicated feature extraction, we can directly use the power spectrum as the inputs for neural networkbased sound source localization [15]. However, unlike in [15], our method employs the real and imaginary parts of the STFT, preserving both the power and phase information. The raw data received by the robot are 4-channel audio signals sampled at 48 kHz.…”

Section: Network Inputmentioning

confidence: 99%

“…The identity mappings in the residual blocks allow a deeper network to be trained without being affected by the vanishing gradients problem. It has been shown that the ResNet is effective for sound source localization problem [15]. The hard parameter sharing in such common trunk provides regularization and reduces the risk of overfitting [17].…”

Section: Network Architecturementioning

confidence: 99%

See 2 more Smart Citations

Joint Localization and Classification of Multiple Sound Sources Using a Multi-task Neural Network

He¹,

Motlíček²,

Odobez³

2018

Interspeech 2018

View full text Add to dashboard Cite

We propose a novel multi-task neural network-based approach for joint sound source localization and speech/non-speech classification in noisy environments. The network takes raw short time Fourier transform as input and outputs the likelihood values for the two tasks, which are used for the simultaneous detection, localization and classification of an unknown number of overlapping sound sources, Tested with real recorded data, our method achieves significantly better performance in terms of speech/non-speech classification and localization of speech sources, compared to method that performs localization and classification separately. In addition, we demonstrate that incorporating the temporal context can further improve the performance.

show abstract

Section: Network Inputmentioning

confidence: 99%

Section: Network Inputmentioning

confidence: 99%

Section: Network Architecturementioning

confidence: 99%

See 1 more Smart Citation

Joint Localization and Classification of Multiple Sound Sources Using a Multi-task Neural Network

He¹,

Motlíček²,

Odobez³

2018

Interspeech 2018

View full text Add to dashboard Cite

show abstract

“…Opposed to the previously described works using refined features directly related to the localization problem, we can also find others using frequency domain features directly [48,52], in some cases generated from spectrograms of general time-frequency representations [51,54]. These approaches represent a step forward compared with the previous ones, as they give the network the responsibility of automatically learn the relationship between spectral cues and the location related information [57] kind of combines both strategies, as they use spectral features but calculating them in a cross-spectral fashion, that is, combining the values from all the available microphones in the so-called Cross Spectral Map (CSM).…”

Section: State Of the Artmentioning

confidence: 99%

Towards End-to-End Acoustic Localization Using Deep Learning: From Audio Signal to Source Position Coordinates

Vera-Diaz¹,

Pizarro²,

Macías-Guarasa³

2018

Preprint

View full text Add to dashboard Cite

This paper presents a novel approach for indoor acoustic source localization using microphone arrays and based on a Convolutional Neural Network (CNN). The proposed solution is, to the best of our knowledge, the first published work in which the CNN is designed to directly estimate the three dimensional position of an acoustic source, using the raw audio signal as the input information avoiding the use of hand crafted audio features. Given the limited amount of available localization data, we propose in this paper a training strategy based on two steps. We first train our network using semi-synthetic data, generated from close talk speech recordings, and where we simulate the time delays and distortion suffered in the signal that propagates from the source to the array of microphones. We then fine tune this network using a small amount of real data. Our experimental results show that this strategy is able to produce networks that significantly improve existing localization methods based on SRP-PHAT strategies. In addition, our experiments show that our CNN method exhibits better resistance against varying gender of the speaker and different window sizes compared with the other methods.

show abstract

“…Significant emerging technologies being discussed in various research studies include wearable technologies, networked and smart environments connected by the Internet of Things, evolving tools, tangible interfaces, human-robot collaborations, processes and interactions, virtual reality, ubiquitous use of machine learning, and deep-learning algorithms [22]. In the last few years, applications based on deep neural networks have performed various tasks, such as speech recognition or image classification, to levels that even exceed human abilities [23][24][25]. SAE is a deep-learning algorithm.…”

Section: Saementioning

confidence: 99%

Smart Shift Decision Method Based on Stacked Autoencoders

Wang

Zhang

et al. 2018

Journal of Control Science and Engineering

View full text Add to dashboard Cite

Manual calibration and testing on real vehicles are common methods of generating shifting schedules for newly developed vehicles. However, these methods are time-consuming. Shifting gear timing is an important operating parameter that affects shifting time, power loss, fuel efficiency, and driver comfort. The stacked autoencoder (SAE) algorithm, a type of artificial neural network, is used in this study to predict shifting gear timing on the basis of throttle percentage, vehicle velocity, and acceleration. Experiments are conducted to obtain training and testing data. Different neural networks are trained with experimental data on a real vehicle under different road conditions collected using the CANcaseXL device and control AMESim simulation model, which was constructed based on real vehicle parameters. The input number of SAE is determined through a comparison between two and three parameters. The output type of SAE is determined through a comparative experiment on pattern recognition and multifitting. Meanwhile, the network structure of SAE is determined through a comparative experiment on simple and deep-learning neural networks. Experimental results demonstrate that using the SAE intelligent shift control strategy to determine shift timing not only is feasible and accurate but also saves time and development costs.

show abstract

Sound Source Localization Using Deep Learning Models

Cited by 112 publications

References 19 publications

Joint Localization and Classification of Multiple Sound Sources Using a Multi-task Neural Network

Joint Localization and Classification of Multiple Sound Sources Using a Multi-task Neural Network

Towards End-to-End Acoustic Localization Using Deep Learning: From Audio Signal to Source Position Coordinates

Smart Shift Decision Method Based on Stacked Autoencoders

Contact Info

Product

Resources

About