Trust in haptic assistance: weighting visual and haptic cues based on error history

Abstract

To effectively interpret and interact with the world, humans weight redundant estimates from different sensory cues to form one coherent, integrated estimate. Recent advancements in physical assistance systems, where guiding forces are computed by an intelligent agent, enable the presentation of augmented cues. It is unknown, however, if cue weighting can be extended to augmented cues. Previous research has shown that cue weighting is determined by the reliability (inversely related to uncertainty) of cues within a trial, yet augmented cues may also be affected by errors that vary over trials. In this study, we investigate whether people can learn to appropriately weight a haptic cue from an intelligent assistance system based on its error history. Subjects held a haptic device and reached to a hidden target using a visual (Gaussian distributed dots) and haptic (force channel) cue. The error of the augmented haptic cue varied from trial to trial based on a Gaussian distribution. Subjects learned to estimate the target location by weighting the visual and augmented haptic cues based on their perceptual uncertainty and experienced errors. With both cues available, subjects were able to find the target with an improved or equal performance compared to what was possible with one cue alone. Our results show that the brain can learn to reweight augmented cues from intelligent agents, akin to previous observations of the reweighting of naturally occurring cues. In addition, these results suggest that the weighting of a cue is not only affected by its within-trial reliability but also the history of errors.

Newly learned shape-colour associations show signatures of reliability-weighted averaging without forced fusion or a memory colour effect

“…Our results add to the growing literature showing that novel cues can be learned and integrated with familiar cues to enhance perception Ernst, 2007;Gibo et al, 2017; 18/02/2022 Shape-colour associations S. Aston 25 Negen et al, 2018Negen et al, , 2021 and extend this finding to the domain of material perception. We have expanded upon these previous findings by testing for a perceptual effect of the novel cue to colour that we introduced.…”

“…Combining independent noisy estimates in this way is the optimal strategy for maximising perceptual precision, as the resulting estimate is less variable (more reliable) than estimates from either single cue alone (Jones, 2016a;Rohde et al, 2016a). Previous work shows that newly-learned novel cues can also be combined with familiar cues to make more precise decisions Ernst, 2007;Gibo et al, 2017;Negen et al, 2018Negen et al, , 2021. For example, a newly-learned audio cue to depth is combined with visual information to improve precision in depth judgements after short-term training (Negen et al, 2018(Negen et al, , 2021.…”

Using Immersive Virtual Reality to Examine How Visual and Tactile Cues Drive the Material-Weight Illusion

“…A potential explanation for this is that when sensory information is combined, the information from one sense can be used to judge the reliability of the other (Atkins et al, 2001). So, the reliability of the visual information may have been judged to be poor because it directly conflicted with information from the more contextually-relevant haptic sense (Gibo et al, 2017). Importantly, this condition revealed the large extent to which haptically-derived expectations drive the MWI.…”

Using Immersive Virtual Reality to Examine How Visual and Tactile Cues Drive the Material-Weight Illusion

“…A potential explanation for this is that when sensory information is combined, the information from one sense can be used to judge the reliability of the other (Atkins et al, 2001 ). So, the reliability of the visual information may have been judged to be poor because it directly conflicted with information from the more contextually-relevant tactile information (Gibo et al, 2017 ). Importantly, this condition provides further evidence for an interaction between modality and higher-level expectations in heaviness perception.…”