Quantifying Facial Expression Intensity and Signal Use in Deaf Signers

Stoll, Chloé; Rodger, Helen; Lao, Junpeng; Richoz, Anne-Raphaëlle; Pascalis, Olivier; Dye, Matthew W.G.; Caldara, Roberto

doi:10.1093/deafed/enz023

Cited by 11 publications

(18 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The results contradict our hypothesis, in which we expected deaf observers to perform better in the recognition of dynamic compared to static expressions, and compared to hearing individuals. A recent study found comparable performance between deaf and hearing observers for the recognition of static stimuli of varying intensities ( Stoll et al., 2019 ). However, previous studies using dynamic stimuli found a dynamic advantage in deaf children when the stimuli did not vary in emotional intensity ( Jones et al., 2017 ), and no advantage was found in deaf adults in a study testing anger and surprise expressions ( Grossman and Kegl, 2007 ).…”

Section: Discussionmentioning

confidence: 99%

“…Recently, using static stimuli, which varied in emotion intensity, Stoll et al (2019) measured the quantity of signal and emotional intensity required to categorise all six basic emotions in deaf signers and hearing non-signers. Recognition performance between the two groups was comparable for all emotions with the exception of disgust, for which deaf signers needed higher levels of signal and intensity for accurate recognition.…”

Section: Introductionmentioning

confidence: 99%

“…Recently, using static stimuli, which varied in emotion intensity, Stoll et al. (2019) measured the quantity of signal and emotional intensity required to categorise all six basic emotions in deaf signers and hearing non-signers.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The recognition of facial expressions of emotion in deaf and hearing individuals

Rodger

Lao

Stoll

et al. 2021

Heliyon

Self Cite

View full text Add to dashboard Cite

During real-life interactions, facial expressions of emotion are perceived dynamically with multimodal sensory information. In the absence of auditory sensory channel inputs, it is unclear how facial expressions are recognised and internally represented by deaf individuals. Few studies have investigated facial expression recognition in deaf signers using dynamic stimuli, and none have included all six basic facial expressions of emotion (anger, disgust, fear, happiness, sadness, and surprise) with stimuli fully controlled for their low-level visual properties, leaving the question of whether or not a dynamic advantage for deaf observers exists unresolved. We hypothesised, in line with the enhancement hypothesis, that the absence of auditory sensory information might have forced the visual system to better process visual (unimodal) signals, and predicted that this greater sensitivity to visual stimuli would result in better recognition performance for dynamic compared to static stimuli, and for deaf-signers compared to hearing non-signers in the dynamic condition. To this end, we performed a series of psychophysical studies with deaf signers with early-onset severe-to-profound deafness (dB loss >70) and hearing controls to estimate their ability to recognize the six basic facial expressions of emotion. Using static, dynamic, and shuffled (randomly permuted video frames of an expression) stimuli, we found that deaf observers showed similar categorization profiles and confusions across expressions compared to hearing controls (e.g., confusing surprise with fear). In contrast to our hypothesis, we found no recognition advantage for dynamic compared to static facial expressions for deaf observers. This observation shows that the decoding of dynamic facial expression emotional signals is not superior even in the deaf expert visual system, suggesting the existence of optimal signals in static facial expressions of emotion at the apex. Deaf individuals match hearing individuals in the recognition of facial expressions of emotion.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The recognition of facial expressions of emotion in deaf and hearing individuals

Rodger

Lao

Stoll

et al. 2021

Heliyon

Self Cite

View full text Add to dashboard Cite

show abstract

“…Indeed, research on intercultural nonverbal communication highlights the importance of nonverbal language in second-language teaching (Yang, 2020). A study by Stoll and colleagues (Stoll et al, 2019) suggests that compared with hearing signers, deaf signers require intensified nonverbal signals to achieve similar emotion recognition accuracies. However, to our knowledge, no studies have examined nonverbal signaling in the absence of spoken language.…”

Section: Discussionmentioning

confidence: 99%

Faces synchronize when communication through spoken language is prevented.

Zhao¹,

Wood²,

Mutlu³

et al. 2023

Emotion

View full text Add to dashboard Cite

Cooperating with another person requires communicating intentions and coordinating behavior. People often accomplish these tasks using spoken language, but verbal communication is not always available. Here, we test the hypothesis that, to establish successful cooperative interaction, people compensate for the temporary loss of one means, verbal communication, by amplifying another, namely nonverbal expressive synchrony. Fifty-seven female dyads, half of whom were prevented from using spoken language, completed four cooperative tasks, two of which induced expressions of emotion, while their faces were filmed. The no-language dyads displayed more facial-expressive synchrony, quantified using a novel application of multidimensional dynamic time warping. We find that solutions to coordinating interaction solved by spoken language can be compensated for by synchronizing facial expressions. The findings also point to one social force-the lack of shared language-that might, in the long-term, select for cultures of increased nonverbal expressiveness and synchrony.

show abstract

“…For instance, profound deafness (but not sign language experience) has been found to enhance visual attention, especially for the periphery (Bavelier et al, 2006; Bosworth & Dobkins, 2002; Pavani & Bottari, 2012). Sign language experience (for both deaf and hearing individuals) enhances performance for face discrimination tasks and shape detection for the lower visual field (Stoll & Dye, 2019; Stoll et al, 2019). The key point is that deafness is not required to exert effects on perceptual gaze control and that being exposed to sign language (not yet producing it) is sufficient to alter perceptual gaze control at a very early age.…”

Section: Discussionmentioning

confidence: 99%

Rapid development of perceptual gaze control in hearing native signing Infants and children

Bosworth

Stone

2021

Developmental Science

View full text Add to dashboard Cite

Children's gaze behavior reflects emergent linguistic knowledge and real-time language processing of speech, but little is known about naturalistic gaze behaviors while watching signed narratives. Measuring gaze patterns in signing children could uncover how they master perceptual gaze control during a time of active language learning. Gaze patterns were recorded using a Tobii X120 eye tracker, in 31 non-signing and 30 signing hearing infants (5-14 months) and children (2-8 years) as they watched signed narratives on video. Intelligibility of the signed narratives was manipulated by presenting them naturally and in video-reversed ("low intelligibility") conditions. This video manipulation was used because it distorts semantic content, while preserving most surface phonological features. We examined where participants looked, using linear mixed models with Language Group (non-signing vs. signing) and Video Condition (Forward vs. Reversed), controlling for trial order. Non-signing infants and children showed a preference to look at the face as well as areas below the face, possibly because their gaze was drawn to the moving articulators in signing space. Native signing infants and children demonstrated resilient, face-focused gaze behavior. Moreover, their gaze behavior was unchanged for video-reversed signed narratives, similar to what was seen for adult native signers, possibly because they already have efficient highly focused gaze behavior. The present study demonstrates that human perceptual gaze control is sensitive to visual language experience over the first year of life and emerges early, by 6 months of age. Results have implications for the critical importance of early visual language exposure for deaf infants.

show abstract

Quantifying Facial Expression Intensity and Signal Use in Deaf Signers

Cited by 11 publications

References 48 publications

The recognition of facial expressions of emotion in deaf and hearing individuals

The recognition of facial expressions of emotion in deaf and hearing individuals

Faces synchronize when communication through spoken language is prevented.

Rapid development of perceptual gaze control in hearing native signing Infants and children

Contact Info

Product

Resources

About