Shape Discrimination Using the Tongue: Implications for a Visual-to-Tactile Sensory Substitution Device

Vincent, Margaret; Tang, Hao; Khoo, Wai L.; Zhu, Zhigang; Ro, Tony

doi:10.1163/22134808-00002542

Cited by 5 publications

(4 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their measured vibration frequency was 170.6 Hz (±5.6) on average, and their mean rise time was 17.8 ms (±0.5) from command to 20% of vibrating amplitude and 32.2 ms (±2.2) from command to 80%. ERM motors are easily available and affordable (e.g., Choi & Kuchenbecker, 2013;van Erp & Self, 2008), which is a major criterion for most applications of vibrotactile displays (e.g., Jones & Sarter, 2008;van Erp & Self, 2008). They have been used in the most recent studies involving 2D tactile pattern recognition (e.g., Barralon et al, 2009;Jones et al, 2009;Kim et al, 2006;Novich & Eagleman, 2015;Schwalk et al, 2015;Yanagida et al, 2004) and more largely for other applications.…”

Section: Apparatusmentioning

confidence: 99%

“…Communicating augmented information through the skin has been a challenge for researchers for about 60 years (Bach-y-Rita, 1972; Geldard, 1957, 1960; see Gallace et al, 2007; Jones & Sarter, 2008, for reviews). Among the options explored over the years, one way of coding information through tactile devices has been to present patterns in two dimensions (2D) to the surface of the skin, which can reproduce visual forms such as printed characters and geometric shapes (e.g., Arnold & Auvray, 2014; Bach-y-Rita et al, 1969; Linvill & Bliss, 1966; Vincent et al, 2016), or be used to convey various contents such as information relating to the user’s context, nearby or distant events, and instructions (e.g., Barralon et al, 2009; Brewster & Brown, 2004; Jones et al, 2009; Schwalk et al, 2015). While a great number of psychophysical studies have been conducted on the perception of 2D tactile patterns in the 70s and 80s (e.g., Bach-y-Rita et al, 1969; Kirman, 1973; White et al, 1970), the subject then progressively fell out of favor despite some unresolved questions and a recent increase of applied studies considering this form of coding.…”

Section: Effectiveness Of Display Modesmentioning

confidence: 99%

See 1 more Smart Citation

Spatiotemporal influences on the recognition of two-dimensional vibrotactile patterns on the abdomen.

Faugloire¹,

Lejeune²,

Rivière³

et al. 2022

Journal of Experimental Psychology: Applied

View full text Add to dashboard Cite

Spatial and temporal factors are known to highly influence tactile perception, but their role has been largely unexplored in the case of two-dimensional (2D) pattern recognition. We investigated whether recognition is facilitated by the spatial and/or temporal separation of pattern elements, or by conditions known to favor perceptual integration, such as the ones eliciting apparent movement. 2D vibrotactile patterns were presented to the abdomen of novice participants. In Experiment 1, we manipulated the spatial (inter-tactor distance) and temporal (burst duration and inter-burst interval) parameters applied to the tracing mode (sequential activation of pattern elements). In Experiment 2, we compared display modes differing in their level of temporal overlap in the presentation of pattern elements: the static mode (simultaneous activation of pattern elements), the slit-scan mode (pattern revealed line by line), and the tracing mode. The results of both experiments reveal that (a) recognition performance increases with the isolation of pattern elements in space and/or in time, (b) spatial and temporal factors interact in pattern recognition, and (c) conditions leading to apparent movement tend to be associated with lower recognition accuracy. These results further our understanding of tactile perception and provide guidance for the design of future vibrotactile communication systems.

show abstract

Section: Apparatusmentioning

confidence: 99%

Section: Effectiveness Of Display Modesmentioning

confidence: 99%

Spatiotemporal influences on the recognition of two-dimensional vibrotactile patterns on the abdomen.

Faugloire¹,

Lejeune²,

Rivière³

et al. 2022

Journal of Experimental Psychology: Applied

View full text Add to dashboard Cite

show abstract

“…However, if tracing is required for a reasonable performance, then tracing with a finger would be easier and faster because finger is more mobile compared to the tongue. Overall, the larger surface of tongue does not seem to provide a useful larger platform because of the non-uniformity in spatial resolution along the tongue (see also Vincent, et al 18 ).…”

Section: Discussionmentioning

confidence: 99%

Poor resolution at the back of the tongue is the bottleneck for spatial pattern recognition

Pamir

Canoluk

Jung

et al. 2020

Sci Rep

View full text Add to dashboard Cite

Spatial patterns presented on the tongue using electro-tactile sensory substitution devices (SSDs) have been suggested to be recognized better by tracing the pattern with the tip of the tongue. We examined if the functional benefit of tracing is overcoming the poor sensitivity or low spatial resolution at the back of the tongue or alternatively compensating for limited information processing capacity by fixating on a segment of the spatial pattern at a time. Using a commercially available SSD, the Brainport, we compared letter recognition performance in three presentation modes; tracing, static, and drawing. Stimulation intensity was either constant or increased from the tip to the back of the tongue to partially compensate for the decreasing sensitivity. Recognition was significantly better for tracing, compared to static and drawing conditions. confusion analyses showed that letters were confused based on their characteristics presented near the tip in static and drawing conditions. the results suggest that recognition performance is limited by the poor spatial resolution at the back of the tongue, and tracing seems to be an effective strategy to overcome this. Compensating for limited information processing capacity or poor sensitivity by drawing or increasing intensity at the back, respectively, does not improve the performance. Visual-to-tactile sensory substitution devices (SSDs) convey visual information to a blind person through touch. Since the seminal work of Bach-y-Rita in the 1960s 1 , tactile representation of visual information has been studied on various body parts including the back 1,2 , abdomen 3-5 , fingers 6-8 , and tongue 9-11. The tongue has been suggested to be a good platform for SSDs because it has higher sensitivity and spatial resolution compared to other body parts 9,12. Users can recognize simple shapes, differentiate a small set of objects, detect motion, point to and avoid high-contrast-staged obstacles when the stimulus is presented sufficiently on the tongue (roughly 30 seconds) 9,13-17. However, navigation and object recognition tasks in real-life situations, where the scene is usually dynamic, may require these tasks to be achieved at shorter times. Vincent, et al. 18 showed that when a stimulus is presented for a short time (0.5 seconds), performance in discriminating basic shapes was not significantly above chance and performance in discriminating different line orientations deteriorated substantially compared to those reported by previous studies. They attributed the better performance in the earlier studies to the longer stimulation durations. This allowed participants to explore the stimulus actively and serially using the tip of the tongue ("tip" from here on), which may be required for spatial recognition through tactile input 19. Our experience with the BrainPort, an FDA-cleared electro-tactile tongue SSD (Wicab Inc., Madison, WI), is consistent with Vincent, et al. 18. The BrainPort delivers the visual information acquired through a head-mounted camera to a 20 × 20 grid...

show abstract

“…The current version of this device, termed the BrainPort, instead uses an eyeglass mounted camera to control patterns of electrical stimulation on the tongue [7]. However, this is expensive for most users and requires extensive training [8], although users can also benefit from customising to their preferences [9]. The spatial resolution of tactile devices is limited by the number of contact points, and the relatively low spatial resolution of the skin.…”

Section: Basic Overview Of Ssds and Issuesmentioning

confidence: 99%

SoundSight: a mobile sensory substitution device that sonifies colour, distance, and temperature

Hamilton-Fletcher

Alvarez

Obrist

et al. 2021

J Multimodal User Interfaces

View full text Add to dashboard Cite

Depth, colour, and thermal images contain practical and actionable information for the blind. Conveying this information through alternative modalities such as audition creates new interaction possibilities for users as well as opportunities to study neuroplasticity. The ‘SoundSight’ App (www.SoundSight.co.uk) is a smartphone platform that allows 3D position, colour, and thermal information to directly control thousands of high-quality sounds in real-time to create completely unique and responsive soundscapes for the user. Users can select the specific sensor input and style of auditory output, which can be based on anything—tones, rainfall, speech, instruments, or even full musical tracks. Appropriate default settings for image-sonification are given by designers, but users still have a fine degree of control over the timing and selection of these sounds. Through utilising smartphone technology with a novel approach to sonification, the SoundSight App provides a cheap, widely accessible, scalable, and flexible sensory tool. In this paper we discuss common problems encountered with assistive sensory tools reaching long-term adoption, how our device seeks to address these problems, its theoretical background, its technical implementation, and finally we showcase both initial user experiences and a range of use case scenarios for scientists, artists, and the blind community.

show abstract

Shape Discrimination Using the Tongue: Implications for a Visual-to-Tactile Sensory Substitution Device

Cited by 5 publications

References 51 publications

Spatiotemporal influences on the recognition of two-dimensional vibrotactile patterns on the abdomen.

Spatiotemporal influences on the recognition of two-dimensional vibrotactile patterns on the abdomen.

Poor resolution at the back of the tongue is the bottleneck for spatial pattern recognition

SoundSight: a mobile sensory substitution device that sonifies colour, distance, and temperature

Contact Info

Product

Resources

About