Speech Recognition for a Humanoid with Motor Noise Utilizing Missing Feature Theory

Nishimura, Yoshitaka; Ishizuka, Mitsuru; Nakadai, Kazuhiro; Nakano, Mikio; Tsujino, Hiroshi

doi:10.1109/ichr.2006.321359

Cited by 19 publications

(20 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their model, however, was unable to deal with ego-motion noise. Nishimura et al [21] estimated the ego noise of distinct gestures and motions of the robot. Using motion commands, the pre-recorded correct noise template matching a recent motion could be selected from the template database and the acoustic features of the aligned template could be used for MFT weight calculation.…”

Section: Related Workmentioning

confidence: 99%

Ego noise cancellation of a robot using missing feature masks

et al. 2011

Self Cite

View full text Add to dashboard Cite

We describe an architecture that gives a robot the capability to recognize speech by cancelling ego noise, even while the robot is moving. The system consists of three blocks: (1) a multi-channel noise reduction block, comprising consequent stages of microphone-array-based sound localization, geometric source separation and post-filtering;(2) a single-channel noise reduction block utilizing template subtraction; and (3) an automatic speech recognition block. In this work, we specifically investigate a missing feature theory-based automatic speech recognition (MFT-ASR) approach in block (3). This approach makes use of spectrotemporal elements derived from (1) and (2) to measure the reliability of the acoustic features, and generates masks to filter unreliable acoustic features. We then evaluated this system on a robot using word correct rates. Furthermore, we present a detailed analysis of recognition accuracy to determine optimal parameters. Implementation of the proposed MFT-ASR approach resulted in significantly higher recognition performance than single or multi-channel noise reduction methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Ego noise cancellation of a robot using missing feature masks

et al. 2011

Self Cite

View full text Add to dashboard Cite

show abstract

“…We introduced a stream weight optimization module which is mentioned in Section II. For AVSR implementation, MFT-based Julius [18] was used.…”

Section: The Second Layer Av Integration Blockmentioning

confidence: 99%

Two-layered audio-visual speech recognition for robots in noisy environments

Yoshida

Nakadai

Okuno

2010

2010 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

Abstract-Audio-visual (AV) integration is one of the key ideas to improve perception in noisy real-world environments. This paper describes automatic speech recognition (ASR) to improve human-robot interaction based on AV integration. We developed AV-integrated ASR, which has two AV integration layers, that is, voice activity detection (VAD) and ASR. However, the system has three difficulties: 1) VAD and ASR have been separately studied although these processes are mutually dependent, 2) VAD and ASR assumed that high resolution images are available although this assumption never holds in the real world, and 3) an optimal weight between audio and visual stream was fixed while their reliabilities change according to environmental changes. To solve these problems, we propose a new VAD algorithm taking ASR characteristics into account, and a linear-regression-based optimal weight estimation method. We evaluate the algorithm for auditoryand/or visually-contaminated data. Preliminary results show that the robustness of VAD improved even when the resolution of the images is low, and the AVSR using estimated stream weight shows the effectiveness of AV integration.

show abstract

“…For a simultaneous speech recognition task of several speakers, Yamamoto et al [7] and Takahashi et al [10] proposed a model for mask generation based on the disturbing effect of leakage noise over speech, because an imperfect source separation causes distorting elements, however their model is unable to deal with ego-motion noise. Nishimura et al [11] estimated ego noises of distinct gestures and motions of the robot. Using motion commands, the pre-recorded correct noise template matching to the recent motion was selected from the template database and the acoustic features of the aligned template are used for MFT weight calculation.…”

Section: A Comparison To Related Workmentioning

confidence: 99%

Multi-talker speech recognition under ego-motion noise using Missing Feature Theory

İnce

Nakadai

Rodemann

et al. 2010

2010 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

Abstract-This paper presents a system that gives a mobile robot the ability to recognize target speaker's speech, even if the robot performs an action and there are multiple speakers talking in the room. Associated problems to this system are twofold: (1) While the robot is moving, the joints inevitably generate ego-motion noise due to its motors. (2) Recognizing target speech against other interfering speech signals is a difficult task. Since typical solutions to (1) and (2), motor noise suppression and sound source separation, both introduce distortion to the processed signals, the performance of automatic speech recognition (ASR) deteriorates. Instead of removing the ego-motion noise with conventional noise suppression methods, in this work, we investigate methods to eliminate the unreliable parts of the audio features that are contaminated by the ego-motion noise. For this purpose, we model masks that filter unreliable speech features based on the ratio of speech and motor noise energies. We analyze the performance of the proposed technique under various test conditions by comparing it to the performance of existing Missing Feature Theory-based ASR implementations. Finally, we propose an integration framework for two different masks that are designed to eliminate ego noise and to filter the leakage energy of interfering sound sources. We demonstrate that the proposed methods achieve a high ASR accuracy.

show abstract

Speech Recognition for a Humanoid with Motor Noise Utilizing Missing Feature Theory

Cited by 19 publications

References 20 publications

Ego noise cancellation of a robot using missing feature masks

Ego noise cancellation of a robot using missing feature masks

Two-layered audio-visual speech recognition for robots in noisy environments

Multi-talker speech recognition under ego-motion noise using Missing Feature Theory

Contact Info

Product

Resources

About