The Supernumerary Hand Illusion in Augmented Reality

Abstract

The classic rubber hand illusion (RHI) experiment studies the sense of embodiment over a fake limb. Distinguished subcomponents of embodiment are ownership (sense of self-attribution of a body), agency (sense of having motor control), and self-location (the spatial experience of being inside a body), and are typically evoked in either reality or virtual reality. In augmented reality (AR), however, visually present real limbs can be augmented with (multiple) fake virtual limbs, which results in a variation of the RHI, the augmented reality supernumerary hand illusion (ARSHI). Such conditions occur, for example, in first-person AR games and in AR-interfaces for tele-robotics. In this article, we examined to what extent humans can experience the sense of embodiment over a supernumerary virtual arm in addition to one or two real arms. We also examine how embodiment is affected by the perceptual visual-tactile synchronicity of the virtual and real limbs, and by the synchronicity of active movement of the virtual and real hand. Embodiment was measured subjectively by questionnaire and objectively by skin conductance responses (SCRs). Questionnaire responses show that ownership, agency, and self-location can be evoked over the virtual arm in the presence of a real arm, and that they are significantly stronger for synchronous conditions than for asynchronous conditions. The perceptual and motorical synchronous condition with three visible hands led to an experience of owning the virtual hand. These responses further show that agency was also strongly experienced over the supernumerary virtual arm, and responses regarding self-location suggest a shift in sensed location when one real arm was in view and an additional location when both real arms where in view. SCRs show no significant effect of condition, but do show a significant habituation effect as a function of the number of conditions performed by participants. When analyzing the relations at the individual participant level between the questionnaire data and skin conductance, we found two clusters of participants: (1) participants with low questionnaire responses and low-medium SCRs and (2) participants with high questionnaire responses and low-high SCRs. Finally, we discuss how virtual hand appearance/realism and willingness to accept virtual limbs could play an important role in the ARSHI, and provide insights on intricacies involved with measuring and evaluating RHIs.

Toward Enhanced Teleoperation Through Embodiment

“…Participants that simultaneously viewed the experimenter's hands from a 1PP and their own hands from a 3PP (using an HMD) while all hands tapped their index fingers, experienced a four-hand illusion (Chen et al, 2018 ). Participants also experienced high levels of ownership and agency over a six-digit virtual hand (Hoyet et al, 2016 ) and a third arm, both in reality (Schaefer et al, 2009 ) and in augmented reality (Rosa et al, 2019 ). In addition, Kishore et al ( 2016 ) found that it is possible to concurrently experience full-body ownership over multiple artificial bodies (two different humanoid robots located in two distinct places and a virtual body represented in immersive VR).…”

Being an Avatar “for Real”: A Survey on Virtual Embodiment in Augmented Reality

“…The previous sections described experiments where the SoE could be evoked with different levels of body avatarization [32], [113], embodiment perspectives [31], [57], environments [21], [22], [59], display types [1], [12], [59], avatar appearances [59], [76], real body visibility [58], and sensory stimulation [20], [31], [127]. We conclude that, to some extent, the SoE can be elicited independently of the technological context as long as the appropriate stimuli and induction conditions are met.…”

“…a sharp knife suddenly falls), studies have shown that participants will attempt to avoid virtual danger in the case where the illusion is successful. This response to danger is reflected in several bodily signals such as the skin conductance response (SCR) and heart rate deceleration (HRD), which can be recorded in both VR and AR [64], [113]. Body ownership illusions have also been correlated with decreases of user body temperature [114] and variations of muscle electrical activity (EMG) [115].…”

“…Examples of objective measurements of virtual hand illusions (VHI). Left: Sensors are placed on the participant's right hand to record their SCR at the moment a virtual knife falls on the embodied hand [113]. Right: EEG recording during VT stimulation in a VHI in AR [20].…”

“…In VR, research has shown that appropriate stimulation can lead users as far as accepting a sixth finger [136] or even a tail added to their virtual body [137]. Rosa et al [113] investigated whether this could also be feasible in AR with the user's real body. Their experiment let participants embody a virtual third arm with a VST HMD, stimulated in a VT and/or VM way.…”