Visual, Haptic, and Bimodal Perception of Size and Stiffness in Virtual Environments

Abstract

Human psychophysical experiments were designed and conducted to investigate the effect of 3D perspective visual images on the visual and haptic perception of size and stiffness in multimodal virtual environments (VEs). Virtual slots of varying length and buttons of varying stiffness were displayed to the subjects, who then were asked to discriminate their size and stiffness respectively using visual and/or haptic cues. The results of the size experiments show that under vision alone, farther objects are perceived to be smaller due to perspective cues and the addition of haptic feedback reduces this visual bias. Similarly, the results of the stiffness experiments show that compliant objects that are farther are perceived to be softer when there is only haptic feedback and the addition of visual feedback reduces this haptic bias. Hence, we conclude that our visual and haptic systems compensate for each other such that the sensory information that comes from visual and haptic channels is fused in an optimal manner.

Visual Feedback Weakens the Augmentation of Perceived Stiffness by Artificial Skin Stretch

“…A potential explanation is that in [7] participants received no visual feedback, whereas in [6] partial visual feedback was provided, supporting our claim that visual feedback may the perceptual illusion. Wu et al [12] found that adding visual displacement information to haptic information reduced a haptic bias to feel more distant objects as softer. Varadharajan et al [11] found no significant contribution of vision to the perception of stiffness magnitude during interaction with haptic-visual conditions.…”

“…In a stiffness magnitude estimation experiment, Varadharajan et al [11] found that the discrimination accuracy of the participants increased by over 20% when combining haptic and visual information relative to relying on just haptic information. In an additional study, Wu et al [12] examined the influence of visual feedback on the perceived stiffness of virtual compliant buttons and concluded that the two senses are fused in an optimal manner. They showed that in the absence of visual feedback, participants compliance estimates were biased such that objects that were farther felt softer.…”

Attribution of sensory prediction error to perception of muscle fatigue

“…Furthermore, sensory prediction error modulates the estimation of the physical properties of external objects or environments. Altering the visual feedback of voluntary movements changes the perceived weight [13][14][15] and dynamics 16 of held objects, and the mechanical impedance of environments 17,18 . These perceptual modulations are interpreted as resulting from attributing the prediction error to changes in the properties of external objects.…”

Multimodal Augmented Reality – Augmenting Auditory-Tactile Feedback to Change the Perception of Thickness

“…We know from elasticity theory [9] that the displacement of a rod or plate that force acts upon depends on the force, elastic modulus of the material, and its thickness. Previous work investigated stiffness perception (i.e., the inverse of elasticity) [3,17] for virtual objects. In this research, we expand this by looking at real objects in an AR setting and investigate the influence of sound and touch feedback upon haptic exploration of thickness.…”