Data-Driven Design of Sound for Enhancing the Perception of Expressive Robotic Movement

Dahl, Luke; Bellona, Jon; Bai, Lin; LaViers, Amy

doi:10.1145/3077981.3078047

Cited by 8 publications

(8 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Detailed results of our study are described in [4]. Here we summarize the findings, which are used to inform the design of our sound synthesis application.…”

Section: Sound Movement Correspondence Findingsmentioning

confidence: 87%

“…This was true for both the qualitative and quantitative analyses. We hypothesize that this may be true because the concept of space does not map very precisely onto the qualities of sounds we studied, and a full discussion is available in [4].…”

Section: Sound Movement Correspondence Findingsmentioning

confidence: 99%

“…This paper documents an application to generate sounds based upon correspondences between movement and sound, so that we may further study the perception of endowing expressive robotic movement with expressively coherent sound. In order to search for these correspondences we conducted an initial study, whose details are presented in [4], and which we summarize here. We created animations of a stick figure performing a simple movement with the qualities of each of the eight BEAs.…”

Section: A Study On Sound Movement Correspondencesmentioning

confidence: 99%

“…In this paper we present a quantitative, empirically-based sound design approach and a synthesis application for generating sounds to accompany various movement types. We summarize a study we conducted [4], whose results suggest a number of correspondences between movement qualities and sound qualities. And we describe how these correspondences are used to control sound synthesis parameters in our application.…”

mentioning

confidence: 93%

“…Generating robotic movement by mapping the components of Laban's Effort system to weighted parameters in an optimal control problem builds on [14]. Our sound synthesis application parameters leverage sound qualities corresponding to movement qualities from an analysis of vocalizations of movement by [4]. These movement to sound quality correspondences will be discussed further in Section 3.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Empirically Informed Sound Synthesis Application for Enhancing the Perception of Expressive Robotic Movement

Bellona¹,

Bai²,

Dahl³

et al. 2017

Proceedings of the 23rd International Conference on Auditory Display - ICAD 2017

Self Cite

View full text Add to dashboard Cite

Since people often communicate internal states and intentions through movement, robots can better interact with humans if they too can modify their movements to communicate changing state. These movements, which may be seen as supplementary to those required for workspace tasks, may be termed “expressive.” However, robot hardware, which cannot recreate the same range of dynamics as human limbs, often limit expressive capacity. One solution is to augment expressive robotic movement with expressive sound. To that end, this paper presents an application for synthesizing sounds that match various movement qualities. Its design is based on an empirical study analyzing sound and movement qualities, where movement qualities are parametrized according to Laban’s Effort System. Our results suggests a number of correspondences between movement qualities and sound qualities. These correspondences are presented here and discussed within the context of designing movement-quality-to-sound-quality mappings in our sound synthesis application. This application will be used in future work testing user perceptions of expressive movements with synchronous sounds.

show abstract

“…Detailed results of our study are described in [4]. Here we summarize the findings, which are used to inform the design of our sound synthesis application.…”

Section: Sound Movement Correspondence Findingsmentioning

confidence: 87%

Section: Sound Movement Correspondence Findingsmentioning

confidence: 99%

Section: A Study On Sound Movement Correspondencesmentioning

confidence: 99%

mentioning

confidence: 93%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Empirically Informed Sound Synthesis Application for Enhancing the Perception of Expressive Robotic Movement

Bellona¹,

Bai²,

Dahl³

et al. 2017

Proceedings of the 23rd International Conference on Auditory Display - ICAD 2017

Self Cite

View full text Add to dashboard Cite

show abstract

The Robot Soundscape

Robinson

Bown

Velonaki

2023

Springer Series on Cultural Computing

View full text Add to dashboard Cite

Perceptual Evaluation of Blended Sonification of Mechanical Robot Sounds Produced by Emotionally Expressive Gestures: Augmenting Consequential Sounds to Improve Non-verbal Robot Communication

Frid

Bresin

2021

Int J of Soc Robotics

View full text Add to dashboard Cite

This paper presents two experiments focusing on perception of mechanical sounds produced by expressive robot movement and blended sonifications thereof. In the first experiment, 31 participants evaluated emotions conveyed by robot sounds through free-form text descriptions. The sounds were inherently produced by the movements of a NAO robot and were not specifically designed for communicative purposes. Results suggested no strong coupling between the emotional expression of gestures and how sounds inherent to these movements were perceived by listeners; joyful gestures did not necessarily result in joyful sounds. A word that reoccurred in text descriptions of all sounds, regardless of the nature of the expressive gesture, was “stress”. In the second experiment, blended sonification was used to enhance and further clarify the emotional expression of the robot sounds evaluated in the first experiment. Analysis of quantitative ratings of 30 participants revealed that the blended sonification successfully contributed to enhancement of the emotional message for sound models designed to convey frustration and joy. Our findings suggest that blended sonification guided by perceptual research on emotion in speech and music can successfully improve communication of emotions through robot sounds in auditory-only conditions.

show abstract

Data-Driven Design of Sound for Enhancing the Perception of Expressive Robotic Movement

Cited by 8 publications

References 11 publications

Empirically Informed Sound Synthesis Application for Enhancing the Perception of Expressive Robotic Movement

Empirically Informed Sound Synthesis Application for Enhancing the Perception of Expressive Robotic Movement

The Robot Soundscape

Perceptual Evaluation of Blended Sonification of Mechanical Robot Sounds Produced by Emotionally Expressive Gestures: Augmenting Consequential Sounds to Improve Non-verbal Robot Communication

Contact Info

Product

Resources

About