A survey on the animation of signing avatars: From sign representation to utterance synthesis

Naert, Lucie; Larboulette, Caroline; Gibet, Sylvie

doi:10.1016/j.cag.2020.09.003

Cited by 26 publications

(17 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the latter case of computer-synthesized motions, developers can use manual or automatic keyframe animation. Each of these engineering approaches has significant benefits and drawbacks (Gibet, 2018;Naert et al, 2020), and each approach has seen significant progress in recent years. Motion capture recordings tend to provide more realistic human movements (Alexanderson & Beskow, 2015;Quandt et al, 2020) but require costly investments of time to process and clean.…”

Section: N R E V I E Wmentioning

confidence: 99%

Attitudes toward signing human avatars vary depending on hearing status, age of signed language exposure, and avatar type

Quandt¹,

Willis²,

Schwenk³

et al. 2021

Preprint

View full text Add to dashboard Cite

The use of virtual humans (i.e., avatars) holds the potential for interactive, automated interaction in domains such as remote communication, customer service, or public announcements. For signed language users, signing avatars could potentially provide accessible content by sharing information in the signer’s preferred or native language. As development of signing avatars has gained traction in recent years, many different methods of creating signing avatars have been developed, and the resulting avatars vary widely in their appearance, the naturalness of their movements, and their facial expressions--all of which may potentially impact users’ acceptance of the avatars. We designed a study to test the effects of these intrinsic properties of different signing avatars, while also examining the extent to which people’s own language experiences change their responses to signing avatars. We created video stimuli showing individual signs produced by 1) a live human signer (Human), 2) an avatar made using computer-synthesized animation (CS Avatar), and 3) an avatar made using high-fidelity motion capture (Mocap avatar). We surveyed 191 American Sign Language users, including Deaf (N = 83), Hard-of-Hearing (N = 34), and Hearing (N= 67) groups. Participants rated the three signers on multiple dimensions which were then combined to form ratings of Attitudes, Impressions, Comprehension, and Naturalness. Analyses demonstrated that the Mocap avatar was rated significantly more positively than the CS avatar on all primary variables. Correlations revealed that signers who acquire sign language later in life are more accepting of, and likely to have positive impressions of signing avatars. Finally, those who learned ASL earlier were more likely to give lower, more negative ratings to the CS avatar, but this association was not seen for the Mocap avatar or the Human signer. Together, these findings suggest that movement quality and appearance significantly impact users’ ratings of signing avatars, and show that signed language users with earlier age of ASL exposure are the most sensitive to movement quality issues seen in computer-generated avatars. We suggest that future efforts to develop signing avatars be considerate of retaining the fluid movement qualities which are integral to signed languages.

show abstract

Section: N R E V I E Wmentioning

confidence: 99%

Attitudes toward signing human avatars vary depending on hearing status, age of signed language exposure, and avatar type

Quandt¹,

Willis²,

Schwenk³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Although research efforts have investigated this problem for over twenty years [3] it is still an open question. With the goal of developing a deeper understanding of the process of converting a textual to a visual language, this article gives a brief overview of the unique aspects of signed languages, reviews the technology underlying avatars and identifies representation features useful for avatar display.…”

Section: Introductionmentioning

confidence: 99%

Sign Language Avatars: A Question of Representation

et al. 2022

View full text Add to dashboard Cite

Given the achievements in automatically translating text from one language to another, one would expect to see similar advancements in translating between signed and spoken languages. However, progress in this effort has lagged in comparison. Typically, machine translation consists of processing text from one language to produce text in another. Because signed languages have no generally-accepted written form, translating spoken to signed language requires the additional step of displaying the language visually as animation through the use of a three-dimensional (3D) virtual human commonly known as an avatar. Researchers have been grappling with this problem for over twenty years, and it is still an open question. With the goal of developing a deeper understanding of the challenges posed by this question, this article gives a summary overview of the unique aspects of signed languages, briefly surveys the technology underlying avatars and performs an in-depth analysis of the features in a textual representation for avatar display. It concludes with a comparison of these features and makes observations about future research directions.

show abstract

“…Virtual human avatars who use signed languages could improve digital infrastructure for accessing information, learning signed languages, or other aspects of signed interactions (Naert et al, 2020 ), especially in situations when face-to-face communication is not possible. Signing avatars could also help disseminate emergency-related information quickly and uniformly throughout a community in the case of evacuations, public health crises, or missing person alerts.…”

Section: Introductionmentioning

confidence: 99%

“…In the latter case of computer-synthesized motions, developers can use manual or automatic keyframe animation. Each of these engineering approaches has significant benefits and drawbacks (Gibet, 2018 ; Naert et al, 2020 ), and each approach has seen significant progress in recent years. Motion capture recordings tend to provide more realistic human movements (Alexanderson and Beskow, 2015 ; Quandt et al, 2020 ) but require costly investments of time to process and clean.…”

Section: Introductionmentioning

confidence: 99%

Attitudes Toward Signing Avatars Vary Depending on Hearing Status, Age of Signed Language Acquisition, and Avatar Type

et al. 2022

View full text Add to dashboard Cite

The use of virtual humans (i.e., avatars) holds the potential for interactive, automated interaction in domains such as remote communication, customer service, or public announcements. For signed language users, signing avatars could potentially provide accessible content by sharing information in the signer's preferred or native language. As the development of signing avatars has gained traction in recent years, researchers have come up with many different methods of creating signing avatars. The resulting avatars vary widely in their appearance, the naturalness of their movements, and facial expressions—all of which may potentially impact users' acceptance of the avatars. We designed a study to test the effects of these intrinsic properties of different signing avatars while also examining the extent to which people's own language experiences change their responses to signing avatars. We created video stimuli showing individual signs produced by (1) a live human signer (Human), (2) an avatar made using computer-synthesized animation (CS Avatar), and (3) an avatar made using high-fidelity motion capture (Mocap avatar). We surveyed 191 American Sign Language users, including Deaf (N = 83), Hard-of-Hearing (N = 34), and Hearing (N = 67) groups. Participants rated the three signers on multiple dimensions, which were then combined to form ratings of Attitudes, Impressions, Comprehension, and Naturalness. Analyses demonstrated that the Mocap avatar was rated significantly more positively than the CS avatar on all primary variables. Correlations revealed that signers who acquire sign language later in life are more accepting of and likely to have positive impressions of signing avatars. Finally, those who learned ASL earlier were more likely to give lower, more negative ratings to the CS avatar, but we did not see this association for the Mocap avatar or the Human signer. Together, these findings suggest that movement quality and appearance significantly impact users' ratings of signing avatars and show that signed language users with earlier age of ASL acquisition are the most sensitive to movement quality issues seen in computer-generated avatars. We suggest that future efforts to develop signing avatars consider retaining the fluid movement qualities integral to signed languages.

show abstract

A survey on the animation of signing avatars: From sign representation to utterance synthesis

Cited by 26 publications

References 44 publications

Attitudes toward signing human avatars vary depending on hearing status, age of signed language exposure, and avatar type

Attitudes toward signing human avatars vary depending on hearing status, age of signed language exposure, and avatar type

Sign Language Avatars: A Question of Representation

Attitudes Toward Signing Avatars Vary Depending on Hearing Status, Age of Signed Language Acquisition, and Avatar Type

Contact Info

Product

Resources

About