Discriminant Features and Temporal Structure of Nonmanuals in American Sign Language

Benitez-Quiroz, C. Fabian; Gökgöz, Kadir; Wilbur, Ronnie Β.; Martı́nez, Aleix M.

doi:10.1371/journal.pone.0086268

Cited by 21 publications

(13 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This sample size is consistent with recent results on the study of emotion (Du et al, 2014). In addition, we analyzed the ASL dataset of Benitez-Quiroz et al (2014), which includes 15 native or near-native signers of ASL. This is the largest annotated set available for the study of non-manual markers in ASL.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

The not face: A grammaticalization of facial expressions of emotion

2016

Self Cite

View full text Add to dashboard Cite

Facial expressions of emotion are thought to have evolved from the development of facial muscles used in sensory regulation and later adapted to express moral judgment. Negative moral judgment includes the expressions of anger, disgust and contempt. Here, we study the hypothesis that these facial expressions of negative moral judgment have further evolved into a facial expression of negation regularly used as a grammatical marker in human language. Specifically, we show that people from different cultures expressing negation use the same facial muscles as those employed to express negative moral judgment. We then show that this nonverbal signal is used as a co-articulator in speech and that, in American Sign Language, it has been grammaticalized as a non-manual marker. Furthermore, this facial expression of negation exhibits the theta oscillation (3–8 Hz) universally seen in syllable and mouthing production in speech and signing. These results provide evidence for the hypothesis that some components of human language have evolved from facial expressions of emotion, and suggest an evolutionary route for the emergence of grammatical markers.

show abstract

Section: Methodsmentioning

confidence: 99%

“…The data used in Experiment 3 is that collected by the authors in a previous study (Benitez-Quiroz et al, 2014). Importantly, the sentences used in Experiments 2 and 3 were the same.…”

Section: Methodsmentioning

confidence: 99%

The not face: A grammaticalization of facial expressions of emotion

2016

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, this process identifies moving the right hand down without any side movement as the most discriminant feature to express and visually recognize the concept (word) “alive.” Additional results are given in Table 2. This is useful, for example, to study grammatical function of behavior in linguistics [4]. …”

Section: Resultsmentioning

confidence: 99%

Labeled Graph Kernel for Behavior Analysis

Zhao

Martı́nez

2016

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

Automatic behavior analysis from video is a major topic in many areas of research, including computer vision, multimedia, robotics, biology, cognitive science, social psychology, psychiatry, and linguistics. Two major problems are of interest when analyzing behavior. First, we wish to automatically categorize observed behaviors into a discrete set of classes (i.e., classification). For example, to determine word production from video sequences in sign language. Second, we wish to understand the relevance of each behavioral feature in achieving this classification (i.e., decoding). For instance, to know which behavior variables are used to discriminate between the words apple and onion in American Sign Language (ASL). The present paper proposes to model behavior using a labeled graph, where the nodes define behavioral features and the edges are labels specifying their order (e.g., before, overlaps, start). In this approach, classification reduces to a simple labeled graph matching. Unfortunately, the complexity of labeled graph matching grows exponentially with the number of categories we wish to represent. Here, we derive a graph kernel to quickly and accurately compute this graph similarity. This approach is very general and can be plugged into any kernel-based classifier. Specifically, we derive a Labeled Graph Support Vector Machine (LGSVM) and a Labeled Graph Logistic Regressor (LGLR) that can be readily employed to discriminate between many actions (e.g., sign language concepts). The derived approach can be readily used for decoding too, yielding invaluable information for the understanding of a problem (e.g., to know how to teach a sign language). The derived algorithms allow us to achieve higher accuracy results than those of state-of-the-art algorithms in a fraction of the time. We show experimental results on a variety of problems and datasets, including multimodal data.

show abstract

“…IV-C). Benitez-Quiroz et al [7] propose a framework that combines linguistic and computational modeling to analyze the discriminant nonmanual features of SL. By applying this framework on five types of sentences of ASL, the experiments reveal that the mouth and teeth features are among the most discriminant non-manual features, in terms of separating conditionals from non-conditionals.…”

Section: Mouth Non-manuals In Existing Aslr Systemsmentioning

confidence: 99%

A survey on mouth modeling and analysis for Sign Language recognition

Antonakos

Roussos

Zafeiriou

2015

2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG)

View full text Add to dashboard Cite

Abstract-Around 70 million Deaf worldwide use Sign Languages (SLs) as their native languages. At the same time, they have limited reading/writing skills in the spoken language. This puts them at a severe disadvantage in many contexts, including education, work, usage of computers and the Internet. Automatic Sign Language Recognition (ASLR) can support the Deaf in many ways, e.g. by enabling the development of systems for Human-Computer Interaction in SL and translation between sign and spoken language. Research in ASLR usually revolves around automatic understanding of manual signs. Recently, ASLR research community has started to appreciate the importance of non-manuals, since they are related to the lexical meaning of a sign, the syntax and the prosody. Nonmanuals include body and head pose, movement of the eyebrows and the eyes, as well as blinks and squints. Arguably, the mouth is one of the most involved parts of the face in non-manuals. Mouth actions related to ASLR can be either mouthings, i.e. visual syllables with the mouth while signing, or non-verbal mouth gestures. Both are very important in ASLR. In this paper, we present the first survey on mouth non-manuals in ASLR. We start by showing why mouth motion is important in SL and the relevant techniques that exist within ASLR. Since limited research has been conducted regarding automatic analysis of mouth motion in the context of ALSR, we proceed by surveying relevant techniques from the areas of automatic mouth expression and visual speech recognition which can be applied to the task. Finally, we conclude by presenting the challenges and potentials of automatic analysis of mouth motion in the context of ASLR.

show abstract

Discriminant Features and Temporal Structure of Nonmanuals in American Sign Language

Cited by 21 publications

References 39 publications

The not face: A grammaticalization of facial expressions of emotion

The not face: A grammaticalization of facial expressions of emotion

Labeled Graph Kernel for Behavior Analysis

A survey on mouth modeling and analysis for Sign Language recognition

Contact Info

Product

Resources

About