Reading with the Tongue: Individual Differences Affect the Perception of Ambiguous Stimuli with the BrainPort

Abstract

There is an increasing interest in non-visual interfaces for HCI to take advantage of the information processing capability of the other sensory modalities. The BrainPort is a vision-to-tactile sensory substitution device that conveys information through electro-stimulation on the tongue. As the tongue is a horizontal surface, it makes for an interesting platform to study the brain's representation of space. But which way is up on the tongue? We provided participants with perceptually ambiguous stimuli and measured how often different perspectives were adopted; furthermore, whether camera orientation and gender had an effect. Additionally, we examined whether personality (trait extraversion and openness) could predict the perspective taken. We found that self-centered perspectives were predominantly adopted, and that trait openness may predict perspective. This research demonstrates how individual differences can affect the usability of sensory substitution devices, and highlights the need for flexible and customisable interfaces.

Vision shapes tactile spatial perspective taking.

“…Future research could systematically vary the stimulated body part, as this could affect which perspective is spontaneously adopted (e.g. for an investigation of spatial perspective-taking for tactile stimuli delivered to the tongue see Richardson, Lloyd-Esenkaya, Petrini, & Proulx, 2020).…”

SoundSight: a mobile sensory substitution device that sonifies colour, distance, and temperature

“…The current version of this device, termed the BrainPort, instead uses an eyeglass mounted camera to control patterns of electrical stimulation on the tongue [7]. However, this is expensive for most users and requires extensive training [8], although users can also benefit from customising to their preferences [9]. The spatial resolution of tactile devices is limited by the number of contact points, and the relatively low spatial resolution of the skin.…”

Spatial Competence and Brain Plasticity in Congenital Blindness via Sensory Substitution Devices

“…However, as all available SSDs suffer from methodological shortcomings from their technology to their adaptability to the environment, it may take a while before we see their widespread use (Chebat et al, 2018a). Current trends investigating the impact of personality traits on SSD use (Richardson et al, 2020) will surely lead to better, more adaptable and customizable devices. Another important question concerns the ideal age to start training with SSDs.…”