Interactive Plants: Multisensory Visual-Tactile Interaction Enhances Emotional Experience

Abstract

Using a multisensory interface system, we examined how people’s emotional experiences change as their tactile sense (touching a plant) was augmented with visual sense (“seeing” their touch). Our system (the Interactive Plant system) senses the electrical capacitance of the human body and visualizes users’ tactile information on a flat screen (when the touch is gentle, the program draws small and thin roots around the pot; when the touch is more harsh or abrupt, big and thick roots are displayed). We contrasted this multimodal combination (touch + vision) with a unimodal interface (touch only or watch only) and measured the impact of the multimodal interaction on participants’ emotion. We found significant emotional gains in the multimodal interaction. Participants’ self-reported positive affect, joviality, attentiveness and self-assurance increased dramatically in multimodal interaction relative to unimodal interaction; participants’ electrodermal activity (EDA) increased in the multimodal condition, suggesting that our plant-based multisensory visual-tactile interaction raised arousal. We suggest that plant-based tactile interfaces are advantageous for emotion generation because haptic perception is by nature embodied and emotional.

Destination Attributes of Memorable Chinese Rural Tourism Experiences: Impact on Positive Arousal, Memory and Behavioral Intention

“…103 Yamauchi et al showed that, compared with simple visual or tactile interaction, tactile and visual multisensory interaction with plants effectively enhanced the arousal level. 104 Therefore, this study speculates that rural tourists either visually appreciate the natural landscape in rural physiography, have intimate contact with the natural areas and flora and fauna through tactile touch, or listen to the various natural sounds, which all trigger emotional arousal. Moreover, the more positive the evaluation of rural physiography experienced through viewing, touching and listening, the stronger the positive arousal will be.…”

“…48 Previous studies have examined the effects of physiography on arousal from the perspective of auditory, visual and tactile sensations. [101][102][103][104] As an important element of nature tourism destinations, the natural soundscape has been widely recognized as an important perceived element and ideal experience for tourists in nature tourism destinations. 101 Jiang studied the connection between the sound landscape in the natural landscape and the emotions of tourists from the perspective of the auditory senses, which shows that the natural soundscape significantly affects the emotional arousal of tourists.…”

Multimodal-Multisensory Experiments

“…Yamauchi et al further demonstrated that people's emotional experiences change as their tactile sense (touching a plant) was augmented with visual sense ("seeing" their touch) in a multisensory interface system [16]. Chen et al (2012) tried to identify possible correlations between increasing levels of cognitive demand and modalities from speech, digital pen, and freehand gesture to eye activity, galvanic skin response, and EEG [17].…”

Multimodal-Multisensory Experiments: Design and Implementation

“…Yamauchi et al combined behavioral measures and multiple mouse motion features to better predict people's emotions and cognitive conflict in computer tasks [12,13]. Yamauchi et al further demonstrated that people's emotional experiences change as their tactile sense (touching a plant) was augmented with visual sense ("seeing" their touch) in a multisensory interface system [14]. Chen et al (2012) tried to identify possible correlations between increasing levels of cognitive demand and modalities from speech, digital pen, and freehand gesture to eye activity, galvanic skin response, and EEG [15].…”