Sound Source Separation Mechanisms of Different Deep Networks Explained from the Perspective of Auditory Perception

Han, Li; Chen, Kean; Wang, Lei; Liu, Jianben; Wan, Baonian; Zhou, Bing

doi:10.3390/app12020832

Cited by 8 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although these proposals are highly accurate, their application is not always feasible in a real-time scenario. This is mainly due to the computational complexity of these approaches being very high, essentially in implementing sound separation mechanisms [42,43].…”

Section: Introductionmentioning

confidence: 99%

Microphone Array for Speaker Localization and Identification in Shared Autonomous Vehicles

et al. 2022

View full text Add to dashboard Cite

With the current technological transformation in the automotive industry, autonomous vehicles are getting closer to the Society of Automative Engineers (SAE) automation level 5. This level corresponds to the full vehicle automation, where the driving system autonomously monitors and navigates the environment. With SAE-level 5, the concept of a Shared Autonomous Vehicle (SAV) will soon become a reality and mainstream. The main purpose of an SAV is to allow unrelated passengers to share an autonomous vehicle without a driver/moderator inside the shared space. However, to ensure their safety and well-being until they reach their final destination, active monitoring of all passengers is required. In this context, this article presents a microphone-based sensor system that is able to localize sound events inside an SAV. The solution is composed of a Micro-Electro-Mechanical System (MEMS) microphone array with a circular geometry connected to an embedded processing platform that resorts to Field-Programmable Gate Array (FPGA) technology to successfully process in the hardware the sound localization algorithms.

show abstract

Section: Introductionmentioning

confidence: 99%

Microphone Array for Speaker Localization and Identification in Shared Autonomous Vehicles

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The possible migration from conventional methods to the methods based on the ANNs in the field of microwave devices was started to be investigated based on the disadvantages, which are mentioned above [28,29]. The usage of ANNs seems promising because ANNs are already successfully used in other fields, such as image processing [30], sound processing [31], and the forecasting of financial markets [32]. The advantage of artificial neural networks (ANNs) in this case is the speed of prediction.…”

Section: Introductionmentioning

confidence: 99%

Trends of Microwave Devices Design Based on Artificial Neural Networks: A Review

et al. 2022

View full text Add to dashboard Cite

The usage of techniques of the artificial neural networks (ANNs) in the field of microwave devices has recently increased. The advantages of ANNs in comparison with traditional full-wave methods are that the prediction speed when the traditional time-consuming iterative calculations are not required and also the complex mathematical model of the microwave device is no longer needed. Therefore, the design of microwave device could be repeated many times in real time. However, methods of artificial neural networks still lag behind traditional full-wave methods in terms of accuracy. The prediction accuracy depends on the structure of the selected neural network and also on the obtained dataset for the training of the network. Therefore, the paper presents a systematic review of the implementation of ANNs in the field of the design and analysis of microwave devices. The guidelines for the systematic literature review and the systematic mapping research procedure, as well as the Preferred Report Items for Systematic Reviews and Meta-Analysis statements (PRISMA) are used to conduct literature search and report the results. The goal of the paper is to summarize the application areas of usage of ANNs in the field of microwave devices, the type and structure of the used artificial neural networks, the type and size of the dataset, the interpolation and the augmentation of the training dataset, the training algorithm and training errors and also to discuss the future perspectives of the usage of ANNs in the field of microwave devices.

show abstract

“…While the learned components of the model attempt to compensate, they are unable to do so. [48,289] recently showed that Conv-TasNet learns to separate signals based on proximity in frequency. In other words, if two sounds are close together in frequency, then Conv-TasNet is more likely to believe that they are from the same source.…”

Section: Toy Analysis Of Filterbank Correlationmentioning

confidence: 99%

Improving Variational Methods for Representation Learning and Generative Modelling of Speech

Braithwaite¹

View full text Add to dashboard Cite

Hearing aids are clinically proven to improve the quality of life of hearing-impaired individuals. However, despite this, many people who could benefit from a hearing aid do not use one. The most common reason for this is that they perform poorly in noisy environments. Machine learning presents a collection of powerful methods for attenuating noise in speech signals. Moreover, state-of-the-art hearing aids are being designed with more powerful processors that can run machine learning algorithms. Several key factors make the deployment of machine learning to hearing aids challenging. Firstly, many state-of-the-art machine learning methods have high resource requirements (e.g., GPUs or large amounts of memory). Hence, these resource-intensive models cannot be run in real-time in low-complexity systems. Secondly, methods need to be causal (i.e., do not depend on access to future information). More generally, an issue is that the objectives that speech enhancement/separation models typically optimise (e.g., L1 or L2) correlate poorly with perceptual quality. The goal of this thesis is to develop improved models for attenuating noise in speech signals. Specifically, we aim to develop more principled models and better understand existing frameworks. Overall, the objective is to develop low-complexity models for attenuating noise in speech signals. We begin by developing an improved autoencoder-based generative model, which we name the Bounded Information Rate Variational Autoencoder. This model resolves the posterior collapse issue with Variational Autoencoders by pre-specifying the information rate between the data and latent variables. In a further extension of this model, we relax the constraints on the distribution of latent variables. Hence, the variance of this distribution is constrained, but not the shape. We name this model the Variance Constrained Autoencoder (VCAE). The VCAE outperforms other state-of-the-art generative models in both reconstruction and generation quality. In subsequent work, we applied the Variance Constrained Autoencoder to speech enhancement. The model is trained to reconstruct a clean target signal from the noisy input. Importantly, rather than enhancing large segments, we enhanced short blocks ($\tilde 4$ ms). This allowed our system to have a lower computation complexity and higher subjective performance than other state-of-the-art approaches. Our final contribution is an empirical study of a speech separation system called Conv-TasNet. This model separates speech signals by applying multiplicative masks to the mixture signal in a learned over-parameterised transform domain. The phenomenon of interest is the structure of the learned transform. Specifically, an over-parameterised frequency domain transform is learned, where more emphasis is placed on the regions with more signal energy. Hence, the transformation resembles something that is perceptually motivated. In the empirical study of Conv-TasNet, we first provide empirical evidence that shows the transform learned by Conv-TasNet prefers uncorrelated coefficients. This explains why a frequency domain transform is learned. Moreover, we show that performance can be improved over the base system by reducing the correlation of the transform space coefficients. In the second part of this contribution, we demonstrate a link between the flatness of the filterbank and the overall separation performance. Furthermore, we show that the density of the vectors affects the flatness of the frame. In other words, we provide evidence that the perceptual structure of the learned filterbank results from an implicit bias in stochastic gradient descent training.

show abstract

Sound Source Separation Mechanisms of Different Deep Networks Explained from the Perspective of Auditory Perception

Cited by 8 publications

References 27 publications

Microphone Array for Speaker Localization and Identification in Shared Autonomous Vehicles

Microphone Array for Speaker Localization and Identification in Shared Autonomous Vehicles

Trends of Microwave Devices Design Based on Artificial Neural Networks: A Review

Improving Variational Methods for Representation Learning and Generative Modelling of Speech

Contact Info

Product

Resources

About