The Skin as a Medium for Sensory Substitution

Abstract

The last 50 years or so has seen great optimism concerning the potential of sensory substitution and augmentation devices to enhance the lives of those with (or even those without) some form of sensory loss (in practice, this has typically meant those who are blind or suffering from low vision). One commonly discussed solution for those individuals who are blind has been to use one of a range of tactile-visual sensory substitution systems that represent objects captured by a camera as outline images on the skin surface in real-time (what Loomis, Klatzky and Giudice, 2012, term general-purpose sensory substitution devices). However, despite the fact that touch, like vision, initially codes information spatiotopically, I would like to argue that a number of fundamental perceptual, attentional, and cognitive limitations constraining the processing of tactile information mean that the skin surface is unlikely ever to provide such general-purpose sensory substitution capabilities. At present, there is little evidence to suggest that the extensive cortical plasticity that has been demonstrated in those who have lost (or never had) a sense can do much to overcome the limitations associated with trying to perceive high rates of spatiotemporally varying information presented via the skin surface (no matter whether that surface be the back, stomach, forehead, or tongue). Instead, the use of the skin will likely be restricted to various special-purpose devices that enable specific activities such as navigation, the control of locomotion, pattern perception, etc.

Biomimetic rehabilitation engineering: the importance of somatosensory feedback for brain–machine interfaces

“…However, artificially re-created sensory percepts run the risk of overloading or distorting natural information processing (Lenay et al 2003) and, in contrast to normal situations, are not constrained by cognitive mechanisms, e.g. attention (Spence 2014). This is one of the most challenging present limitations, in order to control noisy and distracting signals as in natural conditions (reciprocal inhibition).…”

TactiBelt: Integrating Spatial Cognition and Mobility Theories into the Design of a Novel Orientation and Mobility Assistive Device for the Blind

“…However, one of the main constraint of SSD design is the amount of information that can be conveyed to the user without creating an excessive cognitive load [4]. Indeed, the amount of information gathered by our vision is much greater than our tactile or auditory "bandwidth" [5], meaning that SSDs must carefully select what information they convey, and how they encode it.…”

Shape Discrimination Using the Tongue: Implications for a Visual-to-Tactile Sensory Substitution Device

“…However, the BrainPort suffers from limitations, including the high cost (∼US$10 000), cumbersomeness, and its interference with basic functions such as talking and eating (Kendrick, 2009;Steeves and Harris, 2012). Furthermore, some have questioned whether somatosensory input can be utilized for shape discrimination and object identification because of several limitations (Spence, 2014), including the active/serial exploration typically required for object recognition through touch (Gibson, 1962) as well as the limited spatial resolution of the somatosensory system (for a review see Lederman and Klatzky, 2009).…”

“…Given previous claims and findings that object recognition through somatosensory input may be very limited (Spence, 2014) and may require active, serial exploration (Gibson, 1962), we chose to systematically assess the ability of sighted (Experiments 1-4) and blind (Experiment 5) individuals to discriminate static stimuli delivered to the tongue through activation of specific electrodes on the TDU through a computer rather than a head-mounted camera. Previous studies that have allowed active sensing while using SSDs, in which subjects can actively scan and interact with the stimuli (Chebat et al, 2007;Maidenbaum et al, 2016), may have resulted in high levels of performance because of the longer durations of exposure to the stimuli as well as the active sensing itself.…”