Improving human-robot interaction through adaptation to the auditory scene

Abstract

Effective communication with a mobile robot using speech is a difficult problem even when you can control the auditory scene. Robot ego-noise, echoes, and human interference are all common sources of decreased intelligibility. In real-world environments, however, these common problems are supplemented with many different types of background noise sources. For instance, military scenarios might be punctuated by high decibel plane noise and bursts from weaponry that mask parts of the speech output from the robot. Even in non-military settings, however, fans, computers, alarms, and transportation noise can cause enough interference that they might render a traditional speech interface unintelligible. In this work, we seek to overcome these problems by applying robotic advantages of sensing and mobility to a textto-speech interface. Using perspective taking skills to predict how the human user is being affected by new sound sources, a robot can adjust its speaking patterns and/or reposition itself within the environment to limit the negative impact on intelligibility, making a speech interface easier to use.

Dynamically reconfigurable microphone arrays

“…Speech filters, in particular, have received a lot of attention in the field of human-robot interaction for foveating a robot towards a target [5] in 1D. Valin uses specially designed filters for separating out speech signals from ambient and robot noise [1] to identify the angle (yaw, pitch) to a speech source and do auditory streaming.…”

“…For all combinations of 3-5 positions, a combined evidence grid was constructed and a sound source identified. For [3,4,5] mic-pair positions, a total of [142,132,85] grids were constructed and sources extracted across all 8 trials. Figure 5 summarizes the combined results.…”

Using logic to handle conflicts between system, component, and infrastructure goals in complex robotic architectures

“…The challenge here is how to best represent these different types of goals so that they can be compared in a unified way and that conflicts among them can be detected and possibly resolved-in most robotic architectures, infrastructure goals are not compared to agent goals (but see [7] for an exception), and component goals are almost never explicitly represented (but see [10] for an example of a robot's being aware of its auditory environment to improve natural language interactions).…”

Human adaptation to a miniature robot: Precursors of mutual adaptation

“…This results were obtained in an explicit teaching situation. In a more implicit settings, [7] designed a system to help the robot adapt to environmental noise during interaction with humans by controlling its location and orientation. [8] studied the adaptation of a robot manipulator to fuzzy verbal commands using Probabilistic Neural Networks and reported successful learning with a PA-10 redundant manipulator Adaptation of the human to the robot capabilities is less studied.…”