Automatic facial expressions, gaze direction and head movements generation of a virtual agent

Delbosc, Alice; Ochs, Magalie; Ayache, Stéphane

doi:10.1145/3536220.3558806

Cited by 7 publications

(3 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It relies on the frequency components of the movement's speed profile (i.e., changes of speed over time), represented using the Fourier magnitude spectrum. Closely related to this motion invariant but sharing the data‐driven comparison, let us also mention the study of Delbosc et al [DOS*23] which aimed at generating synchronized and believable facial non‐verbal animations for conversational VH. Authors proposed to evaluate their resulting animation in comparison to ground‐truth data both using a distance metrics based on DTW but also comparing jerk results as a good indicator of motion naturalness.…”

Section: Evaluation Methodsmentioning

confidence: 99%

A Survey on Realistic Virtual Human Animations: Definitions, Features and Evaluations

Rekik,

Wuhrer,

Hoyet

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

Generating realistic animated virtual humans is a problem that has been extensively studied with many applications in different types of virtual environments. However, the creation process of such realistic animations is challenging, especially because of the number and variety of influencing factors, that should then be identified and evaluated. In this paper, we attempt to provide a clearer understanding of how the multiple factors that have been studied in the literature impact the level of realism of animated virtual humans, by providing a survey of studies assessing their realism. This includes a review of features that have been manipulated to increase the realism of virtual humans, as well as evaluation approaches that have been developed. As the challenges of evaluating animated virtual humans in a way that agrees with human perception are still active research problems, this survey further identifies important open problems and directions for future research.

show abstract

Section: Evaluation Methodsmentioning

confidence: 99%

A Survey on Realistic Virtual Human Animations: Definitions, Features and Evaluations

Rekik,

Wuhrer,

Hoyet

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…An in-depth study of the corpus combining conversational analysis and data mining methods will be performed to construct the computational behavioral model. In particular, we aim to explore methods for the automatic generation of behavior from corpus as proposed in Cherni et al (2022) and Delbosc et al (2022).…”

Section: The Impact Of Attitude On the Behaviormentioning

confidence: 99%

A forum theater corpus for discrimination awareness

Ochs

Pergandi²,

Ghio³

et al. 2023

Front. Comput. Sci.

View full text Add to dashboard Cite

In this article, we present a new multimodal annotated corpus of forum theater scenes on discrimination awareness. The collected interactions include scenes of ordinary sexism and racism situations played out by different actors in different contexts. The corpus also contains scenes of interactions between an author of discriminatory behavior and a witness trying to make the discriminatory actor aware of their behavior. These confrontations scenes have been played considering different social attitudes (denial, aggressive, and conciliatory). The experimental setup, including motion capture and audio-visual recordings, has been specifically designed to allow the semi-automatic annotation of the corpus and a fine-grained analysis of the multimodal cues associated with social behaviors.

show abstract

“…However, it is important to acknowledge that facial expressions and head movements are inherently interconnected and synchronized with speech [9]. Habibie et al [25] or Delbosc et al [11] introduced an adversarial approach for the automatic generation of facial expressions and head movements jointly. Drawing inspiration from these works, our research focuses on analyzing facial expressions and head movements in a combined manner, with a representation of facial expressions using explainable features, specifically facial action units.…”

Section: Outputs Of the Modelsmentioning

confidence: 99%

Towards the generation of synchronized and believable non-verbal facial behaviors of a talking virtual agent

Delbosc,

Ochs,

Sabouret

et al. 2023

International Cconference on Multimodal Interaction

Self Cite

View full text Add to dashboard Cite

This paper introduces a new model to generate rhythmically relevant non-verbal facial behaviors for virtual agents while they speak. The model demonstrates perceived performance comparable to behaviors directly extracted from the data and replayed on a virtual agent, in terms of synchronization with speech and believability. Interestingly, we found that training the model with two different sets of data, instead of one, did not necessarily improve its performance. The expressiveness of the people in the dataset and the shooting conditions are key elements. We also show that employing an adversarial model, in which fabricated fake examples are introduced during the training phase, increases the perception of synchronization with speech. A collection of videos demonstrating the results and code can be accessed at: https://github.com/aldelb/non_verbal_facial_animation. CCS CONCEPTS• Computing methodologies → Neural networks; Animation.

show abstract

Automatic facial expressions, gaze direction and head movements generation of a virtual agent

Cited by 7 publications

References 39 publications

A Survey on Realistic Virtual Human Animations: Definitions, Features and Evaluations

A Survey on Realistic Virtual Human Animations: Definitions, Features and Evaluations

A forum theater corpus for discrimination awareness

Towards the generation of synchronized and believable non-verbal facial behaviors of a talking virtual agent

Contact Info

Product

Resources

About