Learning to refine behavior using prosodic feedback

Abstract

-We demonstrate the utility of speech prosody as a feedback mechanism in a machine learning system. We have constructed a reinforcement learning system for our humanoid robot Nico, which uses prosodic feedback to refine the parameters of a social waving behavior. We define a waving behavior to be an oscillation of Nico's elbow joint, parameterized by amplitude and frequency. Our system explores a space of amplitude and frequency values, using q-learning to learn the wave which optimally satisfies a human tutor. To estimate tutor feedback in real-time, we first segment speech from ambient noise using a maximum-likelihood voice-activation detector. We then use a k-Nearest Neighbors classifier, with k=3, over 15 prosodic features, to estimate a binary approval/disapproval feedback signal from segmented utterances. Both our voiceactivation detector and prosody classifier are trained on the speech of the individual tutor. We show that our system learns the tutor's desired wave, over the course of a sequence of trialfeedback cycles. We demonstrate our learning results for a single speaker on a space of nine distinct waving behaviors.Index Terms -speech prosody, human-robot interaction, reinforcement learning, socially-guided machine learning.

What prosody tells infants to believe

“…Systems have been built to recognize or describe the prosody of speaker approval (with sustained pitch peak intensity) and prohibition (with low, staccato tones) [15], [9], [8], [11], [10], bids to attract attention (with rising pitch contours) [9], [8], [11], [16], and soothing intent (with falling pitch contours) [8], [11], [17]. Infant-directed prosody has also been investigated for cues to turn-taking [3], speech stream segmentation [4], [5], and new versus old information [6].…”

“…Compared with adult-directed prosody, infant-directed prosody features higher pitch and wider pitch range [4], [18], [19], [20], and longer vowels at phrase [21] and clause boundaries [22]. This prosodic exaggeration has inspired some builders of robotic prosody classifiers to attempt to elicit Motherese-like speech with infant-like robots [8], [10].…”

“…This, in turn, could help reduce learner errors; if the speaker mishears a word as "abul" but the prosody indicates that the item should be familiar, the learner may be able to interpret the misheard word as a familiar word, "apple." Such aids to learning are overlooked in approaches to prosody that account only for information about affect or mood only, as is common in robotic learners [8], [9], [10], [11].…”

Focused word spotting in spoken Korean based on fundamental frequency

“…There are previous studies on prosodic features and word focus of English and Korean. Kim and Scassellati made use of prosodic features of English for robots to learn behavior [7]. Wang and Narayanan studied sentence boundary detection and key information extraction by measuring the speed of English utterances [3,8].…”

Learning to Understand Expressions of Approval and Disapproval through Game-Based Training Tasks

“…Humans express approval and disapproval toward a robot through different channels, such as words, prosody, gestures, facial expressions and touch. Most work on understanding approval and disapproval has been done with single-modal approaches based on prosodic information from speech signals such as intonation, pitch, tempo, loudness and rhythm (Breazeal, 2002) (Kim & Scassellati, 2007). However, we assume that integrating multiple modalities improves the reliability of the recognition and allows the system to adapt to the individual preferences of the user.…”