VisualTouch: Enhancing Affective Touch Communication with Multi-modality Stimulation

Abstract

As one of the most important non-verbal communication channel, touch plays an essential role in interpersonal affective communication. Although some researchers have started exploring the possibility of using wearable devices for conveying emotional information, most of the existing devices still lack the capability to support affective and dynamic touch in interaction. In this paper, we explore the effect of dynamic visual cues on the emotional perception of vibrotactile signals. For this purpose, we developed Visu-alTouch, a haptic sleeve consisting of a haptic layer and a visual layer. We hypothesized that visual cues would enhance the interpretation of tactile cues when both types of cues are congruent. We first carried out an experiment and selected 4 stimuli producing substantially different responses. Based on that, a second experiment was conducted with 12 participants rating the valence and arousal of 36 stimuli using SAM scales. CCS CONCEPTS• Human-centered computing → Haptic devices.

Conceptualising touch in VR

“…This critical role of other senses in achieving touch illusion is central to VR experiences, in particular, the interplay between vision, audio and/or haptic feedback (e.g. Zhang et al 2019). Modal integration was explicitly raised in relation to applications that used visual and sound effects to generate a sense of touch without the use of haptic sensations.…”

Touch without Touching: Overcoming Social Distancing in Semi-Intimate Relationships with SansTouch

“…Another alternative is using wearable devices, for example tactile arm sleeves [53,54], gloves [42], or jackets [10]). These fabricbased wearable devices were compact but expressive, enabling users to exchange afective touch interactions while preserving the possibility of natural interactions with their environment [34,38].…”

“…For example, Multi-Moji [50] used the built-in vibration motors in smartphones to enhance emojis with tactile feedback. TaSSt [24] and VisualTouch [53] used vibration motor arrays embedded on a tactile sleeve to generate tactile patterns on forearms. Similarly, Flex-N-Feel [42] embedded several vibration motors on a pair of gloves to support remote touch communication for couples.…”

“…Combining touch with other modalities could also expand the available range of emotional states and increase the touch perception [43,50,53]. For example, Wilson et al [50] showed that combining visual and thermal modalities in touch interactions allowed users to convey a wider range of afective expressions.…”

“…Based on these insights, we built SansTouch, a multimodal hand sleeve used along with a smartphone to enable handto-hand touch interactions without directly touching. SansTouch combined three modalities that were recommended in the literature to enhance touch sensations: skin-like infation [31,45], heat conditioned to body temperature [3,17,31], and visual feedback [15,50,53]. We then conducted a structured observational study with 12 participants to 1) evaluate user perceptions and the use of SansTouch's handshakes in a face-to-face communication for semiintimate relationships (i.e., colleagues); and 2) elicit user feedback regarding the generalizability aspects of SansTouch, including the use of SansTouch's hand-to-hand interactions for other types of relationships and remote communications.…”