Improving Humanoid Robot Speech Recognition with Sound Source Localisation

Dávila-Chacón, Jorge; Twiefel, Johannes; Liu, Jindong; Wermter, Stefan

doi:10.1007/978-3-319-11179-7_78

Cited by 1 publication

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The experimental setup [33], as shown in Fig. 5, consists of loudspeakers which are placed around the Soundman wooden head [34], behind the white display between 0 • and 180 • along the azimuth plane with the same elevation. The Soundman head is 1.6 meters away from the speakers, designing a far-field communication setup.…”

Section: B Human Robot Simulationmentioning

confidence: 99%

Incorporating End-to-End Speech Recognition Models for Sentiment Analysis

Lakomkin

Zamani

Weber

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

Previous work on emotion recognition demonstrated a synergistic effect of combining several modalities such as auditory, visual, and transcribed text to estimate the affective state of a speaker. Among these, the linguistic modality is crucial for the evaluation of an expressed emotion. However, manually transcribed spoken text cannot be given as input to a system practically. We argue that using ground-truth transcriptions during training and evaluation phases leads to a significant discrepancy in performance compared to real-world conditions, as the spoken text has to be recognized on the fly and can contain speech recognition mistakes. In this paper, we propose a method of integrating an automatic speech recognition (ASR) output with a character-level recurrent neural network for sentiment recognition. In addition, we conduct several experiments investigating sentiment recognition for human-robot interaction in a noise-realistic scenario which is challenging for the ASR systems. We quantify the improvement compared to using only the acoustic modality in sentiment recognition. We demonstrate the effectiveness of this approach on the Multimodal Corpus of Sentiment Intensity (MOSI) by achieving 73,6% accuracy in a binary sentiment classification task, exceeding previously reported results that use only acoustic input. In addition, we set a new state-of-the-art performance on the MOSI dataset (80.4% accuracy, 2% absolute improvement).

show abstract

Section: B Human Robot Simulationmentioning

confidence: 99%