Improving training for sensory augmentation using the science of expertise

How Much Spatial Information Is Lost in the Sensory Substitution Process? Comparing Visual, Tactile, and Auditory Approaches

“…This covers a huge range of design considerations, from utilising the substituting senses’ ability to discriminate and categorise information (e.g., utilising the principles of spatialised hearing—Blauert, 1997, or how “auditory objects” are perceived from an audio stream—Bregman, 1994), a consideration of users’ differing abilities (e.g., early-blind have impaired vertical hearing localisation—Lewald, 2002; Zwiers, Van Opstal, & Cruysberg, 2001; but superior tactile acuity, discrimination of auditory pitch, loudness, and horizontal localisation cues—Fieger, Röder, Teder-Sälejärvi, Hillyard, & Neville, 2006; Goldreich & Kanics, 2003; Gougoux et al., 2004; Kolarik, Cirstea, & Pardhan, 2013; Norman & Bartholomew, 2011; Röder et al., 1999; Wan, Wood, Reutens, & Wilson, 2010), and when appropriate, taking advantage of the users’ multisensory processing biases (e.g., cross-modal correspondences—Deroy, Fasiello, Hayward, & Auvray, 2016; Hamilton-Fletcher et al., 2018; Hamilton-Fletcher, Witzel, Reby, & Ward, 2017; Hamilton-Fletcher, Wright, & Ward, 2016). Beyond this, many other factors remain to assess, such as selecting task-appropriate information; appropriate spatial, temporal, and colour resolutions; avoiding sensory and attentional overloading; training and usability in daily life; and finally, changes in perception, externalisation, and qualia (Auvray, Hanneton, & O'Regan, 2007; Bertram & Stafford, 2016; Brown & Proulx, 2016; Brown, Simpson, & Proulx, 2014, 2015; Hamilton-Fletcher & Ward, 2013; Hartcher-O’Brien & Auvray, 2014; Kristjánsson et al., 2016; Ortiz et al., 2011; Ward & Meijer, 2010).…”

Investigating the Effect of Sensory Concurrency on Learning Haptic Spatiotemporal Signals

“…as key components in the SSAD learning process [4,44], and to-date there are still many opportunities for systematic studies breaking down the individual contributions of these factors in enabling device proficiency [4].…”

“…As human beings we are experts in the visual interpretation of such information, but efforts to accurately interpret haptic feedback have proven a difficult and time-consuming process which requires both intensive training and/or long-term exposure [1,33]. Due to the abstract nature of the information in the signal, there is often a steep learning curve associated with these devices, long-recognized as a barrier to uptake [4,15,23,43].…”

“…However, a broad review of existing training strategies for sensory augmentation devices by Bertram and Stafford has shown that we still lack systematic approaches to teaching information encodings in haptic signals. Specifically, we lack understanding of how feedback and learning conditions contribute to the development of competence and people's ability to effectively interpret encodings in-situ [4].…”

“…To ground the development of our hypothesis, our area of focus is on the role that concurrent sensory information plays in enabling and encouraging such learning. To illustrate, full "sensory concurrency", where duplicate information is received across more than one sense, provides a feedback channel which may be critical for the learning of dynamic haptic signals commonly used by SSADs [4,20]. However, "sensory deprivation", where task-critical senses such as vision are inhibited, forces user attention towards the new signal and creates incentives which also promote learning [32].…”