Development of a robot quizmaster with auditory functions for speech-based multiparty interaction

Abstract

Abstract-This paper presents a robot quizmaster that has auditory functions (i.e., ears) for moderating a multiplayer quiz game. The most basic form of oral interaction in a quiz game is that a quizmaster reads aloud a question, and each player is allowed to answer it whenever the answer comes to his or her mind. A critical problem in such oral interaction is that if multiple players speak almost simultaneously for answering, it is difficult for a "human" quizmaster to recognize overlapping answers and judge the correctness of each answer. To avoid this problem, players have conventionally been required to push a button, raise a hand, or say "Yes" to just get a right to answer a question before doing it. This requirement, however, inhibits natural oral interaction. In this paper we propose a "robot" quizmaster that can identify a player who correctly answers a question first, even when multiple players utter answers almost at the same time. Since our robot uses its own microphones (ears) embedded in the head, individual players are not required to wear small pin microphones close to their mouths. To localize, separate, and recognize overlapping utterances captured by the ears, we use a robot audition software called HARK and an automatic speech recognizer called Julius. Experimental results showed the effectiveness of our approach.

Simultaneous Identification and Localization of Still and Mobile Speakers Based on Binaural Robot Audition

“…Audition is used in various applications of human-robot interaction for performing tasks including scene analysis, speaker identification, speech recognition, speaker localization and tracking [7][8][9][10]. For example, a robot was able to manage a quiz game with multiple players playing at the same time based on their speech utterances in [6]. Further, relying on sound, in [7] an interaction partner was selected among many, and the system adapted its behavior accordingly.…”

“…In [12], a robot performed sound identification, relying on microphone-array-based sound acquisition. Paradigms of sound acquisition include microphone arrays [6,12], and binaurality with the humanoid SIG2 [8], for example. Whatever the context and paradigm, speaker identification and speaker localization -or localization of sound sources in general -have seldom been addressed simultaneously.…”

“…number of tests correctly identified total number of tests ×100. (6) Note that a test can be based on the feature vectors of a single time frame, or on longer durations as in the proposed output combination algorithm in Section 3.3. Regarding the uttered speech, speaker identification can occur in two ways:…”

“…Aside from its biological inspiration, the binaural context allows for a reduction in hardware, using two microphones for tasks that usually require larger microphone arrays. This context notably targets robot audition, which is important for robot operations involving human interaction [1,6,7]. A hearing robot can localize speakers, obtain characteristics of the acoustic scene, and recognize speaker identities and utterances, for example.…”

Automatic Speech Recognition System for Polish Dedicated for a Social Robot

“…Roboty społeczne to roboty przeznaczone do działania razem z człowiekiem w jego codziennym otoczeniu, przy czym ich cechą charakterystyczną jest komunikowanie się z człowiekiem za pomocą sygnałów werbalnych i niewerbalnych. Zagadnienie automatycznego rozpoznawania mowy dotyczy nie tylko robotów społecznych [2][3][4][5], ale m.in. również kontrolowania trajektorii ruchu manipulatorów przemysłowych [6,7] lub pojazdów bezzałogowych [8].…”