Digital Ventriloquism: Giving Voice to Everyday Objects

Iravantchi, Yasha; Goel, Mayank; Harrison, Chris

doi:10.1145/3313831.3376503

Cited by 15 publications

(5 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly to ours, several recent studies delve into guiding users with spatial audio and sonification [2,8,9,16].…”

Section: Related Workmentioning

confidence: 83%

“…As a device that already implements an approximation of HRTFs based on the current user's inter-pupilary distance coupled to inside-out head-tracking, the Microsoft HoloLens 5 was chosen as the main platform for the game. As users call out for objects they need to locate, synthetic localization sounds are virtually propagated by the HoloLens from their positions, in a digital ventriloquism [16] smart-home metaphor, as shown in Fig. 1-bottom.…”

Section: Methodsmentioning

confidence: 99%

“…As the players do not have a previous spatial notion about the environment, digital voice assistant guides the user throughout the experience, reminding them of the recipe, cooking procedures, time constraints and how to find each required ingredient and utensil. The smart home enables sound to be projected from different objects, in a digital ventriloquism manner [16], aiding the user to find the necessary resources without any visual cues. The voice assistant is capable of speech recognition, understanding and replying to the player's questions related to the tasks.…”

Section: Game Scenario and Tasksmentioning

confidence: 99%

See 2 more Smart Citations

Cooking in the dark: a mixed reality empathy experience for the embodiment of blindness

Guarese

Bastidas

Becker

et al. 2021

Proceedings of the 1st XR in Games Workshop (XR in Games 2021)

View full text Add to dashboard Cite

In the context of promoting a sense of empathy for the difference in people without disabilities, we propose a gaming experience that allows users to embody having a visual impairment. By occluding the user’s vision and providing spatialized audio and passive haptic feedback, allied with a speech recognition digital assistant, our goal is to offer a multi-sensory experience to enhance the user’s sense of embodiment inside a mixed reality blindness simulation. Inside the game environment, while expecting a guest to arrive, the player is required to cook a meal completely in the dark. Being aided solely by their remaining senses and a digital assistant, players must go through several tasks as to prepare dinner in time, risking to lose a love interest.

show abstract

“…Similarly to ours, several recent studies delve into guiding users with spatial audio and sonification [2,8,9,16].…”

Section: Related Workmentioning

confidence: 83%

Section: Methodsmentioning

confidence: 99%

Section: Game Scenario and Tasksmentioning

confidence: 99%

See 1 more Smart Citation

Cooking in the dark: a mixed reality empathy experience for the embodiment of blindness

Guarese

Bastidas

Becker

et al. 2021

Proceedings of the 1st XR in Games Workshop (XR in Games 2021)

View full text Add to dashboard Cite

show abstract

“…Indeed, new technologies have given rise to an emergent practice of digital ventriloquism, by which voices are programmed and disembodied in various ways (e.g. smart objects, Iravantchi et al, 2020; deepfakes, Taylor, 2021; lip-syncing, Riszko, 2017). Social media provides a unique space for digital ventriloquism, as the voices portrayed are often disassociated from their origin.…”

Section: Digital Ventriloquism As An Analytical Lensmentioning

confidence: 99%

Digital ventriloquism and celebrity access: Cameo and the emergence of paid puppeteering on digital platforms

Drenten

Psarras

2021

New Media & Society

View full text Add to dashboard Cite

Cameo is part of a growing set of new media platforms trending toward direct routes for monetizing fame. Cameo allows fans to book personalized shout-out videos and provides celebrities—celetoids and reality stars in particular—access to new modes of income, which became increasingly important amid the pandemic. This research explores how the direct monetization of the fan-celebrity relationship is re-shaping the power dynamic of these parasocial relationships. Using digital ventriloquism as an analytical lens to study reality stars (e.g. Real Housewives) on Cameo, this study introduces the concept of paid puppeteering on digital platforms, defined as a form of digital ventriloquism in which a celebrity’s public persona is manipulated and incentivized through financial means on a paid digital platform for the illusion of close parasocial connections with fans. Paid puppeteering reinforces celebrities as gig workers as Cameo mitigates fan access to celebrities—for a fee.

show abstract

“…Recently, the ventriloquism phenomenon became digital ventriloquism, thereby allowing smart speakers to render sound onto everyday objects. This way, the voice agent is not assigned to the given device when it should be contextually and spatially emanated elsewhere (Iravantchi et al, 2020). This phenomenon is interesting, especially when considering binaural audio and the ongoing research on spatial localisation using the head-related transfer (HRT) functions.…”

Section: Introductionmentioning

confidence: 99%

Archives of Acoustics

Mróz,

Kostek

2022

View full text Add to dashboard Cite

This study investigates listeners' perceptual responses in audio-visual interactions concerning binaural spatial audio. Audio stimuli are coupled with or without visual cues to the listeners. The subjective test participants are tasked to indicate the direction of the incoming sound while listening to the audio stimulus via loudspeakers or headphones with the head-related transfer function (HRTF) plugin. First, the methodology assumptions and the experimental setup are described to the participants. Then, the results are presented and analysed using statistical methods. The results indicate that the headphone trials showed much higher perceptual ambiguity for the listeners than when the sound is delivered via loudspeakers. The influence of the visual modality dominates the audio-visual evaluation when loudspeaker playback is employed. Moreover, when the visual stimulus is present, the headphone playback pattern of behavior is not always in response to the loudspeaker playback.

show abstract

Digital Ventriloquism: Giving Voice to Everyday Objects

Cited by 15 publications

References 25 publications

Cooking in the dark: a mixed reality empathy experience for the embodiment of blindness

Cooking in the dark: a mixed reality empathy experience for the embodiment of blindness

Digital ventriloquism and celebrity access: Cameo and the emergence of paid puppeteering on digital platforms

Archives of Acoustics

Contact Info

Product

Resources

About