Evaluating the Influence of Phoneme-Dependent Dynamic Speaker Directivity of Embodied Conversational Agents' Speech

Ehret, Jonathan; Stienen, Jonas; Brozdowski, Chris; Bönsch, Andrea; Mittelberg, Irene; Vorländer, Michael; Kuhlen, Torsten

doi:10.1145/3383652.3423863

Cited by 6 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, some researchers have focused on investigating turn-taking mechanisms in dialogues with the goal of improving human–machine interaction in conversational systems [ 29 ]. For instance, Ehret et al enhanced embodied conversational agents (ECAs) by incorporating non-verbal features such as gestures and gaze to signal turn-taking, thereby making human–machine dialogues smoother and more enjoyable [ 30 ]. In the realm of voice-based human–machine interaction, managing turn-taking in conversations is a crucial area of focus [ 31 , 32 ].…”

Section: Related Workmentioning

confidence: 99%

Sensing the Intentions to Speak in VR Group Discussions

Chen,

Gu,

Zhang

et al. 2024

Sensors

View full text Add to dashboard Cite

While virtual reality (VR) technologies enable remote communication through the use of 3D avatars, it is often difficult to foster engaging group discussions without addressing the limitations to the non-verbal communication among distributed participants. In this paper, we discuss a technique to detect the intentions to speak in group discussions by tapping into intricate sensor data streams from VR headsets and hand-controllers. To this end, we developed a prototype VR group discussion app equipped with comprehensive sensor data-logging functions and conducted an experiment of VR group discussions (N = 24). We used the quantitative and qualitative experimental data to analyze participants’ experiences of group discussions in relation to the temporal patterns of their different speaking intentions. We then propose a sensor-based mechanism for detecting speaking intentions by employing a sampling strategy that considers the temporal patterns of speaking intentions, and we verify the feasibility of our approach in group discussion settings.

show abstract

Section: Related Workmentioning

confidence: 99%

Sensing the Intentions to Speak in VR Group Discussions

Chen,

Gu,

Zhang

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…More recently, Ackermann et al [13] demonstrated that the fluctuations created by the movement of the musicians during solo musical performances are audible both under anechoic and reverberant conditions. Similarly, Ehret et al [14] performed a perceptual evaluation involving static and dynamic phoneme-dependent voice directivities. They showed that participants were not able to distinguish phoneme-dependent directivities from averaged directivities and that their subjective preference might not be dependent on the realism of the directional rendering.…”

Section: Introductionmentioning

confidence: 99%

Perceptual Significance of Tone-Dependent Directivity Patterns of Musical Instruments

Corcuera,

Chatziioannou,

Ahrens

2023

J. Audio Eng. Soc.

View full text Add to dashboard Cite

Musical instruments are complex sound sources that exhibit directivity patterns that not only vary depending on the frequency, but can also change as a function of the played tone. It is yet unclear whether the directivity variation as a function of the played tone leads to a perceptible difference compared to an auralization that uses an averaged directivity pattern. This paper examines the directivity of 38 musical instruments from a publicly available database and then selects three representative instruments among those with similar radiation characteristics (oboe, violin, and trumpet). To evaluate the listeners' ability to perceive a difference between auralizations of virtual environments using tone-dependent and averaged directivities, a listening test was conducted using the directivity patterns of the three selected instruments in both anechoic and reverberant conditions. The results show that, in anechoic conditions, listeners can reliably detect differences between the tone-dependent and averaged directivities for the oboe but not for the violin or the trumpet. Nevertheless, in reverberant conditions, listeners can distinguish tone-dependent directivity from averaged directivity for all instruments under study.

show abstract

“…Similarly, detailed radiation characteristics have been reported for the singing voice [16,[20][21][22]. Furthermore, Ehret et al evaluated the effect of phoneme-dependent directivity implemented for avatars in a virtual space [23]. Although these reports have disclosed the details of the radiation characteristics of the human voice, the presented information does not directly provide the angular resolution required to reproduce the radiation characteristics of the human voice for listening purposes.…”

Section: Introductionmentioning

confidence: 99%

Angular resolution of radiation characteristics required to reproduce uttered speech in all three-dimensional directions

Sugimoto,

Kinoshita

2023

Acoust. Sci. & Tech.

View full text Add to dashboard Cite

For applications in six-degrees-of-freedom (6DoF) content reproduction, this study investigates the angular resolution of radiation characteristics required to reproduce uttered speech in all three-dimensional (3D) directions as an angle at which humans are unable to perceive a difference in the comprehensive audio quality change between adjacent radiation directions. The radiation characteristics of uttered Japanese speech for female and male speakers are measured in three crosssections, namely the horizontal, median, and frontal planes. The measurements are obtained using a spherical jig for 3D microphone arrangement. A subjective evaluation is conducted to clarify the impact of the difference in the radiation directions on the audio quality of uttered speech in each crosssection. Finally, a statistical analysis of the experimental results shows that the required angular resolutions are 30 relative to the horizontal plane, 30 relative to the median plane, and 60 relative to the frontal plane. Therefore, it is concluded that an angular resolution of at least 30 is required to reproduce uttered speech in all 3D directions.

show abstract

Evaluating the Influence of Phoneme-Dependent Dynamic Speaker Directivity of Embodied Conversational Agents' Speech

Cited by 6 publications

References 23 publications

Sensing the Intentions to Speak in VR Group Discussions

Sensing the Intentions to Speak in VR Group Discussions

Perceptual Significance of Tone-Dependent Directivity Patterns of Musical Instruments

Angular resolution of radiation characteristics required to reproduce uttered speech in all three-dimensional directions

Contact Info

Product

Resources

About