Recognition of Tactile Facial Action Units by Individuals Who Are Blind and Sighted: A Comparative Study

Abstract

Given that most cues exchanged during a social interaction are nonverbal (e.g., facial expressions, hand gestures, body language), individuals who are blind are at a social disadvantage compared to their sighted peers. Very little work has explored sensory augmentation in the context of social assistive aids for individuals who are blind. The purpose of this study is to explore the following questions related to visual-to-vibrotactile mapping of facial action units (the building blocks of facial expressions): (1) How well can individuals who are blind recognize tactile facial action units compared to those who are sighted? (2) How well can individuals who are blind recognize emotions from tactile facial action units compared to those who are sighted? These questions are explored in a preliminary pilot test using absolute identification tasks in which participants learn and recognize vibrotactile stimulations presented through the Haptic Chair, a custom vibrotactile display embedded on the back of a chair. Study results show that individuals who are blind are able to recognize tactile facial action units as well as those who are sighted. These results hint at the potential for tactile facial action units to augment and expand access to social interactions for individuals who are blind.

Emotional Response to Vibrothermal Stimuli

“…Embr wave is a wrist-worn device that warms or cools to provide thermal stimulation in the range of +/−4 degrees Celsius from average skin temperature in a safe and comfortable form factor. The Haptic Chair [34] is a custom-built haptic device consisting of an ergonomic mesh chair, the back of which is embedded with a two-dimensional array of small vibrating ERM motors that convey subtle low-amplitude vibrations. A simple graphical user interface (GUI) for this study was implemented using python, as shown in Figure 2.…”

“…We already know the affective impact of thermal stimulation [7]; hence, we hypothesize that there is value in exploring the emotional response to higher-order vibrothermal (vibrotactile and thermal) stimulation. We used the Haptic Chair, introduced in [34], together with an Embr Wave bracelet, to generate vibrothermal patterns.…”

Individual differences in spontaneous facial expressions in people with visual impairment and blindness

“…Furthermore, there has been less attention paid to individual differences in people who share the same disability category, visual disability. In addition, many prior studies were merely conducted with a small sample of participants with visual disabilities (Chiesa et al, 2015;Cole et al, 1989;Freedman, 1964;Galati et al, 2001;Gao et al, 2013;Iverson & Goldin-Meadow, 1997;McDaniel et al, 2019;Webb, 1977;Zhao et al, 2018) and focused on children with visual disabilities over adults with visual disabilities (Ellis et al, 1987;Fraiberg, 1975;Fulcher, 1942;Galati et al, 2003;Ghosh, 2014;Goodenough, 1932;Iverson & Goldin-Meadow, 1997, 1998Thompson, 1941;Webb, 1977). To address the knowledge gap, this study focuses on individual differences in adults with visual disabilities showing spontaneous facial expressions of emotions.…”