Data-driven and physical model-based designs of probabilistic spatial dictionary for online meeting diarization and adaptive beamforming

Ito, Nobuyasu; Araki, Shoko; Nakatani, Tomohiro

doi:10.23919/eusipco.2017.8081391

Cited by 7 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To be able to initialize with values for the class-dependent parameters one often needs external knowledge such as the approximate source positions [84]. It is also possible to distribute the initial values randomly in their domain of definition, e.g., in [17,Page 33] complex Watson mode vectors were drawn from a uniform distribution on the surface of the complex unit hypersphere.…”

Section: Initializationmentioning

confidence: 99%

Integration of Neural Networks and Probabilistic Spatial Models for Acoustic Blind Source Separation

Drude

Haeb‐Umbach

2019

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

Despite a lot of progress in speech separation, enhancement, and automatic speech recognition realistic meeting recognition is still fairly unsolved. Most research on speech separation either focuses on spectral cues to address single-channel recordings or spatial cues to separate multichannel recordings and exclusively either rely on neural networks or probabilistic graphical models. Integrating a spatial clustering approach and a deep learning approach using spectral cues in a single framework can significantly improve automatic speech recognition performance and improve generalizability given that a neural network profits from a vast amount of training data while the probabilistic counterpart adapts to the current scene. This thesis at hand, therefore, concentrates on the integration of two fairly disjoint research streams, namely single-channel deep learning-based source separation and multi-channel probabilistic modelbased source separation. It provides a general framework to integrate spatial and spectral cues in which neural networks and probabilistic graphical models complement each other in achieving state of the art performance in blind source separation on noisy, reverberant data. The efficacy of the proposed approaches is evaluated on simulated artificial mixtures as well as real recordings of simultaneously active speakers. The key findings are (1) a cascade integration in which a neural network initializes a probabilistic graphical model provides substantial improvement, (2) spatial cues can be used for unsupervised training of neural networks, (3) tight integration, an integration in which a joint agreement between both modalities and models is found, leads to lowest word error rates and best generalizability to unseen real mixtures.

show abstract

Section: Initializationmentioning

confidence: 99%

Integration of Neural Networks and Probabilistic Spatial Models for Acoustic Blind Source Separation

Drude

Haeb‐Umbach

2019

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

show abstract

“…Though accurate localization is expected in this case, training data collection would be cumbersome. For a general test environment, we consider the direct sound propagation vector [33] as the spatial centroid instead. This vector depends on the array geometry only and is given by:…”

Section: Weight Parameter Estimationmentioning

confidence: 99%

Target Speaker Localization Based on the Complex Watson Mixture Model and Time-Frequency Selection Neural Network

Wang

Yan

2018

Applied Sciences

View full text Add to dashboard Cite

Common sound source localization algorithms focus on localizing all the active sources in the environment. While the source identities are generally unknown, retrieving the location of a speaker of interest requires extra effort. This paper addresses the problem of localizing a speaker of interest from a novel perspective by first performing time-frequency selection before localization. The speaker of interest, namely the target speaker, is assumed to be sparsely active in the signal spectra. The target speaker-dominant time-frequency regions are separated by a speaker-aware Long Short-Term Memory (LSTM) neural network, and they are sufficient to determine the Direction of Arrival (DoA) of the target speaker. Speaker-awareness is achieved by utilizing a short target utterance to adapt the hidden layer outputs of the neural network. The instantaneous DoA estimator is based on the probabilistic complex Watson Mixture Model (cWMM), and a weighted maximum likelihood estimation of the model parameters is accordingly derived. Simulative experiments show that the proposed algorithm works well in various noisy conditions and remains robust when the signal-to-noise ratio is low and when a competing speaker exists. components, especially the direct sound components, in the observed signals [9]. These techniques include those based on the signal power [10], the coherent-to-diffuse ratio [11], and the speech presence probability [12].The motivation behind time-frequency weighting is the sparsity assumption of the signal spectra, or in other words, the assumption that only one source is active in each time frequency bin [13,14]. As suggested in [15], even if the signal is severely corrupted by noises or interferences, there exist target-dominant time-frequency regions that are sufficient enough for localization. It is also found that the human auditory system may perform source separation jointly with source localization [16]. The same motivation lies behind some fairly recent studies that perform robust source localization based on time-frequency masking [17][18][19], in which the authors propose the usage of Deep Neural Networks (DNNs) to perform mask estimation first. The masks are used to weight the narrowband estimates, and the combined algorithms achieve superior performance in adverse environments. Given that mask estimation is a well-defined task in monaural speech separation and large progress has been made with DNN [20], the combination of mask and the SRP-PHAT (or the MUSIC) algorithm is natural and straightforward. Using neural networks, there are also other practices to localize speech sources directly through spatial classification [21,22].The methods discussed above are directly applicable to TSL if the target speaker is the only speech source in the environment. When a competing speaker exits, multiple source localization techniques [23] could be applied; however, the localization results remain non-discriminative, and further post-processing for speaker identification is needed. TSL in a multi-speaker environ...

show abstract

“…However, the estimation of TDOA is sensitive to reverberation and noise. An alternate formulation based on a probabilistic spatial dictionary and Watson mixture modeling of directional features is proposed in [8]. A pretrained (data-driven) or pre-computed (physics based) spatial dictionary is used, which limits the application of the method to a finite set of source positions and known microphone geometry.…”

Section: Introductionmentioning

confidence: 99%

Latent variable approach to diarization of audio recordings using ad-hoc randomly placed mobile devices

Chetupalli,

Bhowmick,

Sreenivas

2018

Preprint

View full text Add to dashboard Cite

Diarization of audio recordings from ad-hoc mobile devices using spatial information is considered in this paper. A twochannel synchronous recording is assumed for each mobile device, which is used to compute directional statistics separately at each device in a frame-wise manner. The recordings across the mobile devices are asynchronous, but a coarse synchronization is performed by aligning the signals using acoustic events, or real-time clock. Direction statistics computed for all the devices, are then modeled jointly using a Dirichlet mixture model, and the posterior probability over the mixture components is used to derive the diarization information. Experiments on real life recordings using mobile phones show a diarization error rate of less than 14%.

show abstract

Data-driven and physical model-based designs of probabilistic spatial dictionary for online meeting diarization and adaptive beamforming

Cited by 7 publications

References 15 publications

Integration of Neural Networks and Probabilistic Spatial Models for Acoustic Blind Source Separation

Integration of Neural Networks and Probabilistic Spatial Models for Acoustic Blind Source Separation

Target Speaker Localization Based on the Complex Watson Mixture Model and Time-Frequency Selection Neural Network

Latent variable approach to diarization of audio recordings using ad-hoc randomly placed mobile devices

Contact Info

Product

Resources

About