Development and evaluation of a new virtual reality-based audio-tactile cueing-system to guide visuo-spatial attention

Knobel, Samuel E. J.; Gyger, Nathan T.; Nyffeler, Thomas; Cazzoli, Dario; Müri, René M.; Nef, Tobias

doi:10.1109/embc44109.2020.9176516

Cited by 3 publications

(8 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While respective modes were chosen more often than VV and AV, somewhat unexpected, the AA mode was comparably popular. This is different from findings that revealed a high usability for combined AT cues compared to auditory cues alone [53]. To further enhance the effectiveness of the AA mode, it might be helpful to choose more distinguishable timbres for auditory proximity and transition cues [101].…”

Section: Mode Preferencementioning

confidence: 59%

Multisensory Proximity and Transition Cues for Improving Target Awareness in Narrow Field of View Augmented Reality Displays

Trepkowski

Marquardt

Eibich

et al. 2022

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Fig. 1: Demonstration of proximity and transition cues in an augmented reality city information system scenario. Being aware of newly emerging, yet potentially important information can become difficult in dynamic environments. Proximity cues (left side) provide spatial information in either the visual or the auditory modality and help users to become aware of out-of-view objects ("target"). When the target crosses the border of the field of view of the visual output device, the proximity cue is replaced by a transition cue (right side). A short visual, auditory or tactile notification signals the user that the target entered the field of view and is potentially visible.

show abstract

Section: Mode Preferencementioning

confidence: 59%

Multisensory Proximity and Transition Cues for Improving Target Awareness in Narrow Field of View Augmented Reality Displays

Trepkowski

Marquardt

Eibich

et al. 2022

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

“…Additionally, for the application of the tactile cue, a special cushion as an inlay piece for the HMD was used, as described by Knobel et al [ 29 ]. The cushion contains 6 coin vibrators symmetrically distributed over the forehead.…”

Section: Methodsmentioning

confidence: 99%

“…The tactile cue (1-second vibration) was applied using punctual vibration that came from a special inlay of the HMD that contained 6 symmetrically positioned individually controllable coin vibrators [ 29 ]. The coin vibrators were controlled in a way that mimicked the spatial behavior of sound.…”

Section: Methodsmentioning

confidence: 99%

“…Indeed, the effects of auditory cues, presented in a VR setting, on visuospatial attention deployment have been successfully explored in neglect patients and show promising results [ 27 , 28 ]. In contrast, to the best of our knowledge, no study has so far investigated the effects of tactile cues in VR on the spatial deployment of visual attention in neglect patients, although results obtained in healthy controls seem encouraging [ 29 , 30 ]. Finally, previous results in 2D settings suggest that multimodal cueing (ie, combining auditory and tactile cueing at the same time) may result in superior effects than single cueing [ 29 , 31 ].…”

Section: Introductionmentioning

confidence: 99%

“…In contrast, to the best of our knowledge, no study has so far investigated the effects of tactile cues in VR on the spatial deployment of visual attention in neglect patients, although results obtained in healthy controls seem encouraging [ 29 , 30 ]. Finally, previous results in 2D settings suggest that multimodal cueing (ie, combining auditory and tactile cueing at the same time) may result in superior effects than single cueing [ 29 , 31 ]. However, the effects of this combination for patients with visual attention deficits in a 3D VR setting have not been studied.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Effects of Virtual Reality–Based Multimodal Audio-Tactile Cueing in Patients With Spatial Attention Deficits: Pilot Usability Study

Knobel¹,

Kaufmann²,

Geiser³

et al. 2022

JMIR Serious Games

Self Cite

View full text Add to dashboard Cite

Background Virtual reality (VR) devices are increasingly being used in medicine and other areas for a broad spectrum of applications. One of the possible applications of VR involves the creation of an environment manipulated in a way that helps patients with disturbances in the spatial allocation of visual attention (so-called hemispatial neglect). One approach to ameliorate neglect is to apply cross-modal cues (ie, cues in sensory modalities other than the visual one, eg, auditory and tactile) to guide visual attention toward the neglected space. So far, no study has investigated the effects of audio-tactile cues in VR on the spatial deployment of visual attention in neglect patients. Objective This pilot study aimed to investigate the feasibility and usability of multimodal (audio-tactile) cueing, as implemented in a 3D VR setting, in patients with neglect, and obtain preliminary results concerning the effects of different types of cues on visual attention allocation compared with noncued conditions. Methods Patients were placed in a virtual environment using a head-mounted display (HMD). The inlay of the HMD was equipped to deliver tactile feedback to the forehead. The task was to find and flag appearing birds. The birds could appear at 4 different presentation angles (lateral and paracentral on the left and right sides), and with (auditory, tactile, or audio-tactile cue) or without (no cue) a spatially meaningful cue. The task usability and feasibility, and 2 simple in-task measures (performance and early orientation) were assessed in 12 right-hemispheric stroke patients with neglect (5 with and 7 without additional somatosensory impairment). Results The new VR setup showed high usability (mean score 10.2, SD 1.85; maximum score 12) and no relevant side effects (mean score 0.833, SD 0.834; maximum score 21). A repeated measures ANOVA on task performance data, with presentation angle, cue type, and group as factors, revealed a significant main effect of cue type (F30,3=9.863; P<.001) and a significant 3-way interaction (F90,9=2.057; P=.04). Post-hoc analyses revealed that among patients without somatosensory impairment, any cue led to better performance compared with no cue, for targets on the left side, and audio-tactile cues did not seem to have additive effects. Among patients with somatosensory impairment, performance was better with both auditory and audio-tactile cueing than with no cue, at every presentation angle; conversely, tactile cueing alone had no significant effect at any presentation angle. Analysis of early orientation data showed that any type of cue triggered better orientation in both groups for lateral presentation angles, possibly reflecting an early alerting effect. Conclusions Overall, audio-tactile cueing seems to be a promising method to guide patient attention. For instance, in the future, it could be used as an add-on method that supports attentional orientation during established therapeutic approaches.

show abstract

Effects of Virtual Reality–Based Multimodal Audio-Tactile Cueing in Patients With Spatial Attention Deficits: Pilot Usability Study (Preprint)

Knobel¹,

Kaufmann²,

Geiser³

et al. 2021

Preprint

View full text Add to dashboard Cite

BACKGROUND Virtual reality (VR) devices are increasingly being used in medicine and other areas for a broad spectrum of applications. One of the possible applications of VR involves the creation of an environment manipulated in a way that helps patients with disturbances in the spatial allocation of visual attention (so-called hemispatial neglect). One approach to ameliorate neglect is to apply cross-modal cues (ie, cues in sensory modalities other than the visual one, eg, auditory and tactile) to guide visual attention toward the neglected space. So far, no study has investigated the effects of audio-tactile cues in VR on the spatial deployment of visual attention in neglect patients. OBJECTIVE This pilot study aimed to investigate the feasibility and usability of multimodal (audio-tactile) cueing, as implemented in a 3D VR setting, in patients with neglect, and obtain preliminary results concerning the effects of different types of cues on visual attention allocation compared with noncued conditions. METHODS Patients were placed in a virtual environment using a head-mounted display (HMD). The inlay of the HMD was equipped to deliver tactile feedback to the forehead. The task was to find and flag appearing birds. The birds could appear at 4 different presentation angles (lateral and paracentral on the left and right sides), and with (auditory, tactile, or audio-tactile cue) or without (no cue) a spatially meaningful cue. The task usability and feasibility, and 2 simple in-task measures (performance and early orientation) were assessed in 12 right-hemispheric stroke patients with neglect (5 with and 7 without additional somatosensory impairment). RESULTS The new VR setup showed high usability (mean score 10.2, SD 1.85; maximum score 12) and no relevant side effects (mean score 0.833, SD 0.834; maximum score 21). A repeated measures ANOVA on task performance data, with presentation angle, cue type, and group as factors, revealed a significant main effect of cue type (F30,3=9.863; P<.001) and a significant 3-way interaction (F90,9=2.057; P=.04). Post-hoc analyses revealed that among patients without somatosensory impairment, any cue led to better performance compared with no cue, for targets on the left side, and audio-tactile cues did not seem to have additive effects. Among patients with somatosensory impairment, performance was better with both auditory and audio-tactile cueing than with no cue, at every presentation angle; conversely, tactile cueing alone had no significant effect at any presentation angle. Analysis of early orientation data showed that any type of cue triggered better orientation in both groups for lateral presentation angles, possibly reflecting an early alerting effect. CONCLUSIONS Overall, audio-tactile cueing seems to be a promising method to guide patient attention. For instance, in the future, it could be used as an add-on method that supports attentional orientation during established therapeutic approaches.

show abstract

Development and evaluation of a new virtual reality-based audio-tactile cueing-system to guide visuo-spatial attention

Cited by 3 publications

References 23 publications

Multisensory Proximity and Transition Cues for Improving Target Awareness in Narrow Field of View Augmented Reality Displays

Multisensory Proximity and Transition Cues for Improving Target Awareness in Narrow Field of View Augmented Reality Displays

Effects of Virtual Reality–Based Multimodal Audio-Tactile Cueing in Patients With Spatial Attention Deficits: Pilot Usability Study

Effects of Virtual Reality–Based Multimodal Audio-Tactile Cueing in Patients With Spatial Attention Deficits: Pilot Usability Study (Preprint)

Contact Info

Product

Resources

About