Robust Speech Recognition Based on Dereverberation Parameter Optimization Using Acoustic Model Likelihood

Gómez, Randy; Kawahara, Tatsuya

doi:10.1109/tasl.2010.2052610

Cited by 25 publications

(26 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are many successful designs focusing on the optimisation of dereverberation parameters based on feedback obtained from ASR results (Gomez, Kawahara, 2010;Seltzer et al, 2004). …”

Section: Dereverberation Methodsmentioning

confidence: 99%

Two-Microphone Dereverberation for Automatic Speech Recognition of Polish

Kundegorski¹,

Jackson²,

Ziółko³

2015

Archives of Acoustics

View full text Add to dashboard Cite

Reverberation is a common problem for many speech technologies, such as automatic speech recognition (ASR) systems. This paper investigates the novel combination of precedence, binaural and statistical independence cues for enhancing reverberant speech, prior to ASR, under these adverse acoustical conditions when two microphone signals are available. Results of the enhancement are evaluated in terms of relevant signal measures and accuracy for both English and Polish ASR tasks. These show inconsistencies between the signal and recognition measures, although in recognition the proposed method consistently outperforms all other combinations and the spectral-subtraction baseline.

show abstract

“…There are many successful designs focusing on the optimisation of dereverberation parameters based on feedback obtained from ASR results (Gomez, Kawahara, 2010;Seltzer et al, 2004). …”

Section: Dereverberation Methodsmentioning

confidence: 99%

Two-Microphone Dereverberation for Automatic Speech Recognition of Polish

Kundegorski¹,

Jackson²,

Ziółko³

2015

Archives of Acoustics

View full text Add to dashboard Cite

show abstract

“…However, this assumption may not hold true; thus, we show later an optimization process aimed to further strengthen the assumption in the wavelet wavelet-based dereverberation optimized for asr ) and B( f , w). Consequently, the recovered early reflection is processed via Cepstral Mean Normalization (CMN) [6] prior to the ASR. From this point onward, we assume that processing is conducted in framewise manner, dropping the index w.…”

Section: A) Model For Dereverberationmentioning

confidence: 99%

Optimized wavelet-domain filtering under noisy and reverberant conditions

Gómez

Kawahara

Nakadai

2015

SIP

Self Cite

View full text Add to dashboard Cite

The paper addresses a robust wavelet-based speech enhancement for automatic speech recognition in reverberant and noisy conditions. We propose a novel scheme in improving the speech, late reflection, and noise power estimates from the observed contaminated signal. The improved estimates are used to calculate the Wiener gain in filtering the late reflections and additive noise. In the proposed scheme, optimization of the wavelet family and its parameters is conducted using an acoustic model (AM). In the offline mode, the optimal wavelet family is selected separately for the speech, late reflections, and background noise based on the AM likelihood. Then, the parameters of the selected wavelet family are optimized specifically for each signal subspace. As a result we can use a wavelet sensitive to the speech, late reflection, and the additive noise, which can independently and accurately estimate these signals directly from an observed contaminated signal. For speech recognition, the most suitable wavelet is identified from the pre-stored wavelets, and wavelet-domain filtering is conducted to the noisy and reverberant speech signal. Experimental evaluations using real reverberant data demonstrate the effectiveness and robustness of the proposed method.In a real-world enclosed environment, the speech signal is reflected and arrives at different time delays when observed by the microphone. This effect is considered as a form of a contamination due to channel distortion, and commonly known as reverberation. The degree of reverberation depends on the reverberation time T 60 , which dictates the severity of distortion. Speech contamination is one of the most common problems in automatic speech recognition (ASR) applications. In the perspective of ASR, any form of contamination of the speech signal at runtime (test condition) is a mismatch to the acoustic model (AM) (training condition). The mismatch may result in the degradation of the ASR performance. Thus, speech enhancement is one of the most important topics in robust ASR. In this paper, we focus primarily on the topic of dereverberation for ASR; since background noise is always present in a real environment, we address enhancement in reverberant and noisy condition, and extend our dereverberation framework to include denoising effect.The scheme of decomposition of the reverberant signal into early and late reflections [1] simplifies the treatment of reverberation. In this scheme, the late reflection

show abstract

“…Depending on the size of the training dataset, the number of Gaussian mixture components are increased to improve signal discrimination. In our system, a total of 256 Gaussian mixture components are used for each model and the training of the two GMM classes is based on the Expectation-Maximization algorithm [5]. The microphone array-processed data is windowed using a 25ms frame.…”

Section: ) Speaker Diarizationmentioning

confidence: 99%

“…The system is scalable and portable, and employs established and state-of-the-art techniques in speech [5], vision [6], and graphics processing (GPU) for multiple-people multimodal interaction data capture and analysis. It consists of a large display equipped with multiple sensing devices spaced on a portable structure: one microphone array, six HD video cameras, and two depth sensors (see Fig.…”

Section: Introductionmentioning

confidence: 99%

Multiparty Interaction Understanding Using Smart Multimodal Digital Signage

Tung

Gómez

Kawahara

et al. 2014

IEEE Trans. Human-Mach. Syst.

Self Cite

View full text Add to dashboard Cite

Abstract-This paper presents a novel multimodal system designed for multi-party human-machine interaction understanding. The design of human-computer interfaces for multiple users is challenging because simultaneous processing of actions and reactions have to be consistent. The proposed system consists of a large display equipped with multiple sensing devices: microphone array, HD video cameras, and depth sensors. Multiple users positioned in front of the panel freely interact using voice or gesture while looking at the displayed content, without wearing any particular devices (such as motion capture sensors or head mounted devices). Acoustic and visual information is captured and processed jointly using established and state-of-the-art techniques to obtain individual speech and gaze direction. Furthermore, a new framework is proposed to model A/V multimodal interaction between verbal and nonverbal communication events. Dynamics of audio signals obtained from speaker diarization and head poses extracted from video images are modeled using hybrid dynamical systems (HDS). We show that HDS temporal structure characteristics can be used for multimodal interaction level estimation, which is useful feedback that can help to improve multi-party communication experience. Experimental results using synthetic and real-world datasets of group communication such as poster presentations show the feasibility of the proposed multimodal system.

show abstract

Robust Speech Recognition Based on Dereverberation Parameter Optimization Using Acoustic Model Likelihood

Cited by 25 publications

References 25 publications

Two-Microphone Dereverberation for Automatic Speech Recognition of Polish

Two-Microphone Dereverberation for Automatic Speech Recognition of Polish

Optimized wavelet-domain filtering under noisy and reverberant conditions

Multiparty Interaction Understanding Using Smart Multimodal Digital Signage

Contact Info

Product

Resources

About