An Evaluation of a Wearable Assistive Device for Augmenting Social Interactions

Abstract

Gaze behaviors contain rich information regarding a person's emotions and engagements. Reciprocal eye contact can invoke feelings of liking between two strangers. But blind people cannot perceive and establish the eye contact with sighted counterparts, causing their feelings of social isolation and low confidence in conversations. Thus, our research purpose is to let blind people perceive and react gaze behaviors in social interactions. A Social Glasses system has been implemented iteratively to deliver the multisensory feedback channels of the "eye contact", integrating both visual and tactile feedback. Specifically, the system consists of a Social Glasses device and a tactile wristband, which are worn by a blind person. The Social Glasses simulates the natural gaze for the blind person, aiming at establishing the "eye contact" between blind and sighted people. The tactile wristband enables the blind person to perceive the corresponding tactile feedback when an "eye contact" happens. To test the system, we conducted a user experiment with 40 participants, including 10 blind-sighted pairs (N=20) and 10 blindfolded-sighted pairs (N=20), to see how it could help increase the communication quality between blind and sighted people, as well as to suggest implications for its design. Our main findings demonstrated that both the simulated gaze and the tactile feedback were significantly effective to enhance the communication quality in blind-sighted conversations. Overall, we contribute (1) empirical research findings on how a Social Glasses system enhances the communication quality in blind-sighted conversations; and (2) design principles to inform future assistive wearable device for augmenting social interactions. INDEX TERMS Accessibility, artificial gaze, assistive technology, communication quality, eye tracking, face-to-face communication, gaze contact, visual impairments.

Investigating socially assistive systems from system design and evaluation: a systematic review

“…They were often applied with computer vision technologies to capture visual cues and provide haptic or audio feedback. A typical form is using smart glasses alone ([ 14 , 42 , 47 , 53 , 59 , 61 ]) or with smart belts ([ 45 , 49 , 50 ]). Smart bands and watches are also very common solutions ([ 57 , 62 , 63 ]).…”

“…Sensor and device network technologies were applied in 11 studies. Apart from the multi-device systems that were used with wearable technologies ([ 14 , 39 , 44 , 61 ]), a large proportion of the solutions were distributed or multi-agent systems that were developed mainly based on a client/server architecture. The client devices could interact with each other online or offline, and their status could also reflect on the server device.…”

“…Face and eye behavior A total of 14 studies ([ 14 , 41 – 45 , 49 – 53 , 57 , 83 , 84 ]) explored Socially Assistive Systems to perceive face and eye behavior of interaction partners. Among them, the majority of these systems supported identifying face behavior.…”

“…In many cases, studies involving assistive technologies focus on enabling fundamental capabilities or “survival” skills of specific user groups (e.g., facilitating visually impaired users to navigate a digital map ([ 9 , 10 ]), or increasing the mobility of people with physical disabilities ([ 11 , 12 ])). Relatively fewer studies ([ 13 , 14 ]) have focused on the assistive technologies’ values of enhancing target users’ social interaction qualities. Nonetheless, neither people nor technology can exist in a social vacuum [ 15 ].…”

“…Through this way, older adults can maintain social connectedness with their caregivers. Other examples can be found regarding Socially Assistive Systems for people with disabilities, such as a haptic display that can convey facial movements to help blind people understand interaction partners’ facial expressions and emotions through vibration feedback [ 23 ]; a wearable device that allows blind people to perceive and react to gaze from conversation partners through haptic and visual feedback ([ 14 , 24 ]); and a nonverbal communication application that supports real-time social distance regulation for children with ASD [ 25 ].…”