Semiotic Analysis of Robot Sounds in Films: Implications for Sound Design in Social Robotics

Abstract

This paper investigates the sound design of robots in films and their potential influence on the field of social robotics. Cinematic robot portrayals have inspired researchers and practitioners in Human-Robot Interaction (HRI). While the non-verbal sounds of iconic film robots like R2-D2 and Wall-E have been explored, this study takes a more comprehensive approach. We explore a broader selection of 15 films featuring humanoid robots across decades through a semiotic analysis of their non-verbal communication sounds, including those related to movements and internal mechanisms . Informed by Bateman and Schmidt’s multimodal film analysis framework, which draws on Saussure’s organization of signs through paradigmatic and syntagmatic relations, our analysis interprets the paradigmatic axis as an examination of the sound and the syntagmatic axis as an exploration of the events surrounding the sound. The findings uncover two primary film robot sound materials: mechanical and synthetic. Furthermore, the analysis revealed several narrative themes, categorized into two groups based on the syntagmatic focus: sounds associated with the robots’ visual appearances and sounds linked to the broader elements within the scene. The discussion section explores the implications of these findings for social robotics, including the importance of sound materials, the role of movement sounds in communication and emotional expression, and the significance of narrative and context in human-robot interaction. The paper also acknowledges the challenges in translating film sound design into practical applications in social robotics. This study provides valuable insights for HRI researchers, practitioners, and sound designers seeking to enhance non-verbal auditory expressions in social robots.