A multistage approach to blind separation of convolutive speech mixtures

Jan, Tariqullah; Wang, Wenwu; Wang, DeLiang

doi:10.1016/j.specom.2011.01.002

Cited by 24 publications

(18 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…IdBM techniques often introduce musical noise, caused by errors in the estimation of the time-frequency masks and manifested in isolated T-F units. A number of techniques have been proposed to suppress musical noise distortions introduced by IdBM techniques [32,33]. While musical noise might be distracting to the listeners, it has not been found to be detrimental in terms of speech intelligibility.…”

Section: Relationship Between Proposed Residual Constraints and Tmentioning

confidence: 99%

Reasons why Current Speech-Enhancement Algorithms do not Improve Speech Intelligibility and Suggested Solutions

Loizou

Kim

2011

IEEE Trans. Audio Speech Lang. Process.

234

134

View full text Add to dashboard Cite

Existing speech enhancement algorithms can improve speech quality but not speech intelligibility, and the reasons for that are unclear. In the present paper, we present a theoretical framework that can be used to analyze potential factors that can influence the intelligibility of processed speech. More specifically, this framework focuses on the fine-grain analysis of the distortions introduced by speech enhancement algorithms. It is hypothesized that if these distortions are properly controlled, then large gains in intelligibility can be achieved. To test this hypothesis, intelligibility tests are conducted with human listeners in which we present processed speech with controlled speech distortions. The aim of these tests is to assess the perceptual effect of the various distortions that can be introduced by speech enhancement algorithms on speech intelligibility. Results with three different enhancement algorithms indicated that certain distortions are more detrimental to speech intelligibility degradation than others. When these distortions were properly controlled, however, large gains in intelligibility were obtained by human listeners, even by spectral-subtractive algorithms which are known to degrade speech quality and intelligibility.

show abstract

Section: Relationship Between Proposed Residual Constraints and Tmentioning

confidence: 99%

Reasons why Current Speech-Enhancement Algorithms do not Improve Speech Intelligibility and Suggested Solutions

Loizou

Kim

2011

IEEE Trans. Audio Speech Lang. Process.

234

134

View full text Add to dashboard Cite

show abstract

“…The AV atoms are inseparable, i.e., each audio atom and its associated visual atom always appear in pairs at a TS position of the AV sequence. As a result, if a visual atom appears at the TS position , i.e., , there exists a corresponding none-zero coefficient in the set , subject to (2) In the above set, rounds a number to its nearest integer; denotes the same temporal position as with a finer resolution. The coarse TS position and comprise a fine TS position .…”

Section: A Generative Modelmentioning

confidence: 99%

“…To optimise it, first we need to calculate the overall matching criterion using (5), with being tied with via set (2). In the -th iteration of the coding stage, the optimal atom index and the associated translation can therefore be found by maximizing the following objective function: (7) where is associated with as defined in set (2). Then we can set values in the parameter set : (8) Finally, the residual 2 will be updated via: (9) There are iterations in total.…”

Section: ) New Matching Criterionmentioning

confidence: 99%

“…This is known as the cocktail party problem [1]. To replicate such capabilities with machines is however extremely challenging, and a popular method for addressing this challenge is to model the problem under the framework of blind source separation (BSS), where the mixtures are usually assumed to be convolutive (or reverberant), in order to model the surface reflections of sounds in an enclosed room environment [2]. Many algorithms have been developed to recover the original unknown source signals from such mixtures [3]- [12] Independent component analysis (ICA) [3]- [6] exploits the statistical independence between source signals to address the BSS problem.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Source Separation of Convolutive and Noisy Mixtures Using Audio-Visual Dictionary Learning and Probabilistic Time-Frequency Masking

Liu

Wang

Jackson

et al. 2013

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

In existing audio-visual blind source separation (AV-BSS) algorithms, the AV coherence is usually established through statistical modelling, using e.g., Gaussian mixture models (GMMs). These methods often operate in a low-dimensional feature space, rendering an effective global representation of the data. The local information, which is important in capturing the temporal structure of the data, however, has not been explicitly exploited. In this paper, we propose a new method for capturing such local information, based on audio-visual dictionary learning (AVDL). We address several challenges associated with AVDL, including cross-modality differences in size, dimension and sampling rate, as well as the issues of scalability and computational complexity. Following a commonly employed bootstrap coding-learning process, we have developed a new AVDL algorithm which features, a bimodality balanced and scalable matching criterion, a size and dimension adaptive dictionary, a fast search index for efficient coding, and cross-modality diverse sparsity. We also show how the proposed AVDL can be incorporated into a BSS algorithm. As an example, we consider binaural mixtures, mimicking aspects of human binaural hearing, and derive a new noise-robust AV-BSS algorithm by combining the proposed AVDL algorithm with Mandel's BSS method, which is a state-of-the-art audio-domain method using time-frequency masking. We have systematically evaluated the proposed AVDL and AV-BSS algorithms, and show their advantages over the corresponding baseline methods, using both synthetic data and visual speech data from the multimodal LILiR Twotalk corpus.Index Terms-Audio-visual coherence, blind source separation, convolutive mixtures, dictionary learning, noisy mixtures. 1053-587X © 2013 IEEE

show abstract

“…Pitch information is relatively reliable under noisy conditions and can be used to improve the system performance [81]. Another potential direction is to use the property of the sources and noise/interferences, such as sparseness, to facilitate the identification of the reliable regions within the mixture that can be used to estimate the sources [74][75][76][77]. This is mainly due to the following three reasons.…”

Section: Future Directionsmentioning

confidence: 99%