Toward the manipulation of time and space in extended reality: a preliminary study on multimodal Tau and Kappa illusions in the visual-tactile domain

Abstract

In the last few years, Extended reality (XR) has enabled novel forms of sensory experiences and social interplay, which can be hardly experienced in real life. However, the full potential of XR has not been exploited yet, since vision remains the main interaction modality, and the time-and spacemodulation of the sense of self -which could open interesting perspectives in several scenarios -is still largely unexplored. To pave the path to a multi-modal manipulation of the sense of time and space in immersive XR, in this work we discuss the preliminary outcomes of the first investigation in the visualtactile domain of two well known perceptual illusions affecting spatial and temporal perception, i.e. Tau and Kappa effects, respectively. We compared the effects originated from unimodal stimulation (i.e., only visual or tactile) with the same effects induced by convergent bimodal stimulation (i.e., visual and tactile), delivered to the forearm. Results show that both Tau and Kappa effects are affected by the multi-modality of the stimulation, and that the perceptual bias differently affects time-or space-perception based on the modality used for stimulus delivery. Our results, although preliminary, seem to suggest that multimodal perceptual illusions could be a viable solution for time-and space-modulation of the sense of self in immersive XR and advanced social human-robot interaction.

When space rules time: the Kappa effect with concurrent visual and tactile stimulation

“…In VR, we speculate that the tactile stimuli should improve the sense of immersion while supporting the Kappa effect driven by the visual domain. Another interesting point regards the attention: as opposed to previous experiments 3,5 where participants were alternately asked about time or space, in this study participants were instructed to constantly focus on the intervals' duration. As reported by Polti et al 36 paying attention to time resulted in a lengthened perceived duration, whereas diverting attention away from time shortened it.…”

“…Despite its relevance in the neuroscience of human perception, only a few studies have investigated the Kappa effect in vision and somatosensation: in particular, Cai et al 3 reported an increase in the Kappa effect by providing visual feedback to bimodal audio-tactile stimulation, therefore supporting the hypothesis that the Kappa effect is dominated by the high spatial acuity of vision. In our former preliminary study, the interaction between the visual (V) and tactile (T) domains for the elicitation of the Kappa effect has been investigated in the forearm through visuo-tactile stimuli finding similar outcomes 5 . Here, we comprehensively characterize the link between time perception and multimodal visual-tactile conditions, also through psychometric functions, within the peripersonal space (PPS) (i.e., at the palm and the forearm).…”

“…Several studies reported on such inter-dependencies in the space-time perception by testing the visual, auditory and tactile modalities and finding experimental results highly dependent on the modality and on the tasks performed in the experiment 3,4 . To this end, the authors in a previous study 5 compared both the Tau and Kappa effects elicited by concurrent visual-tactile stimulation using a shared setup and complementary experimental tasks on space and time perception. Results showed that the Tau effect was only seldom elicited especially in the visual modality, in agreement with recent findings 6 .…”

Altering Time Perception in Virtual Reality Through Multimodal Visual-Tactile Kappa Effect

“…Furthermore, the Kappa effect elicited through multimodal stimulation has received little attention. This is especially true for what concerns concurrent visual-tactile stimulation [13].…”

“…In our previous works, we pioneered the investigation of time-space illusions elicited by multimodal visual-tactile stimuli delivered through physical interfaces in the real world. In a preliminary study [13] we compared the Kappa effect in the peripersonal space (PPS) using a wearable device equipped with LEDs and vibrotactile actuators. The study compared unimodal visual (V), tactile (T) and bimodal (VT) stimuli suggesting the presence of the Kappa effect in all modalities.…”