Embedding a Panoramic Representation of Infrared Light in the Adult Rat Somatosensory Cortex through a Sensory Neuroprosthesis

Hartmann, Konstantin; Thomson, Eric E.; Zea, Ivan; Yun, Richy; Mullen, Peter; Canarick, Jay; Huh, Albert; Nicolelis, Miguel A. L.

doi:10.1523/jneurosci.3285-15.2016

Cited by 28 publications

(44 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Interestingly, when the directionality of the ICMS was reversed such that stimulation occurred when moving away from the platform (μ = 180°), rat Sa reached a performance plateau statistically indistinguishable from the visual cue baseline (t[46] = −0.96; P = 0.345). First, this result demonstrates reversal learning of our encoding paradigm, whereby after first learning one ICMS contingency, the animal had the capacity to remap the appropriate behavioral response to a new ICMS contingency (26). Second, it shows that even when ICMS encoded the platform direction with very low acuity, the rat learned to locate the platform with the same proficiency as with a visual cue.…”

Section: Resultsmentioning

confidence: 71%

“…Prior work has shown that rats can learn to discriminate between 4 potential goal locations on the basis of ICMS delivered when their heading coincides with the direction of the true goal location (35). Several variations on this result were subsequently reported (26,36,37). Our sensory BMI experiment also used a searching task with goal direction feedback.…”

Section: Discussionmentioning

confidence: 76%

“…Importantly, we also observed different learned active sensing strategies: looping for low-acuity feedback and zigzagging for high-acuity feedback. Prior studies investigated reversal learning by changing the encoding function, but only at the level of performance (26,36). Second, we identified an optimal strategy as a function of the encoding parameters, which involved precise turning at ICMS transitions.…”

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Learning active sensing strategies using a sensory brain–machine interface

Richardson

Ghenbot

Liu

et al. 2019

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Diverse organisms, from insects to humans, actively seek out sensory information that best informs goal-directed actions. Efficient active sensing requires congruity between sensor properties and motor strategies, as typically honed through evolution. However, it has been difficult to study whether active sensing strategies are also modified with experience. Here, we used a sensory brain–machine interface paradigm, permitting both free behavior and experimental manipulation of sensory feedback, to study learning of active sensing strategies. Rats performed a searching task in a water maze in which the only task-relevant sensory feedback was provided by intracortical microstimulation (ICMS) encoding egocentric bearing to the hidden goal location. The rats learned to use the artificial goal direction sense to find the platform with the same proficiency as natural vision. Manipulation of the acuity of the ICMS feedback revealed distinct search strategy adaptations. Using an optimization model, the different strategies were found to minimize the effort required to extract the most salient task-relevant information. The results demonstrate that animals can adjust motor strategies to match novel sensor properties for efficient goal-directed behavior.

show abstract

Section: Resultsmentioning

confidence: 71%

Section: Discussionmentioning

confidence: 76%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Learning active sensing strategies using a sensory brain–machine interface

Richardson

Ghenbot

Liu

et al. 2019

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…Building up a corresponding computational theory will deepen our understanding about the extensibility of the central nervous system and the organism-machine hybrid intelligence. It seems clear from experimental results that we have not come close to exhausting the potential for incorporating novel sources of information into cortical processing modes 14 , this also raises a very interesting question: can we know or prove that we can already perceive all the dimensions of the world?…”

Section: Discussionmentioning

confidence: 99%

A Mathematical Theory of Cortex-Receptor Artificial Extension

Xing

2020

Sci Rep

View full text Add to dashboard Cite

Many physiology experiments demonstrate that an organism's cortex and receptor system can be artificially extended, giving the organism new types of perceptual capabilities. To examine artificial extension of the cortex-receptor system, I propose a computational model that allows new types of sensory pathways to be added directly to the computational model itself in an online manner. A synapse expandable artificial neuron model that can grow new synapses, forming a bridge between the novel perceptual information and the existing neural network is introduced to absorb the novel sensory pathway. The experimental results show that the computational model can effectively integrate sudden emerged sensory channels and the neural circuits in the computational model can be reused for novel modalities without influencing the original modality. Many physiological experiments have demonstrated the expandability of an organism's cortex-receptor system. Human L-pigment gene knock-in mice, which express a human long-wavelength-sensitive cone photopigment, acquired a new capacity for chromatic discrimination 1. Rats that receive information from the infrared environment in their somatosensory cortex can perceive infrared light 2. Monkeys have learned to discriminate artificial tactile stimuli in an active tactile exploration task, where a brain-machine-brain interface was used to deliver the stimuli to their primary somatosensory cortex 3. The rat and monkey experiments imply that the sensing capability of an organism can be expanded artificially throughout the organism's lifetime, differing from the natural way, where organisms acquire new experiences through genetic variation and those sensory transducers are fixed at birth 4,5. To provide a mathematical theory for the cortex-receptor artificial extension, in this study, I build a computational model that can adapt to a novel sensory pathway in an online manner. According to the experiments 1-3 , I divide the extension into two situations: (i) emergence of novel sensory receptors in an existing perceptual channel (Fig. 1a) and (ii) emergence of novel perceptual channels in the whole system (Fig. 1b). Inspired by the phenomenon that novel experience can induce a formation of new spines in the brain 6,7 , a synapse expandable artificial neuron model that can grow new synapses to absorb extended sensory signal during learning is designed. The model is different from the classic artificial neuron model whose synapse structure is fixed 8 , which implies a non-extendibility for the novel sensory signals. Meanwhile, to enable the extension of the sensory pathway in the above two situations, a hierarchical and modularized computational structure inspired by the brain structure 9 is designed. The synapse expandable artificial neuron can work on any level of the computational structure, which provides an adequate theoretical explanation to the cortex-receptor artificial extension. Results Computational framework. The computational framework for situation (i) (Fig. 2a) and situation (ii...

show abstract

“…However, a number of open questions remain, such as the optimal timescale or distribution of timescales for adaptation and whether the degree of adaptation must be matched to the properties of the specific neurons being stimulated. Work in primates 6,32 and rats 33 suggests that the plasticity of the brain will allow even a few channels of stimulation to become effective at providing a rich sensory experience, and complex spatiotemporal coding 34 with enough bandwidth to be clinically useful.…”

Section: Discussionmentioning

confidence: 99%

Creating a neuroprosthesis for active tactile exploration of textures

O’Doherty

Shokur

Medina

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

39Intracortical microstimulation (ICMS) of the primary somatosensory cortex (S1) can 40 produce percepts that mimic somatic sensation and thus has potential as an approach to 41 sensorize prosthetic limbs. However, it is not known whether ICMS could recreate active texture 42 exploration-the ability to infer information about object texture by using one's fingertips to scan 43 a surface. Here we show that ICMS of S1 can convey information about the spatial frequencies 44 of invisible virtual gratings through a process of active tactile exploration. Two rhesus monkeys 45 scanned pairs of visually identical screen objects with the fingertip of a hand avatar, controlled 46 via a joystick and later via a brain-machine interface, to find the one with denser virtual gratings. 47The gratings consisted of evenly spaced ridges that were signaled through ICMS pulses 48 generated when the avatar's fingertip crossed each ridge. The monkeys learned to interpret 49 these ICMS patterns evoked by the interplay of their voluntary movements and the virtual 50 textures of each object. Discrimination accuracy across a range of grating densities followed 51Weber's law of just-noticeable differences (JND), a finding that matches normal cutaneous 52 sensation. Moreover, one monkey developed an active scanning strategy where avatar velocity 53 was integrated with the ICMS pulses to interpret the texture information. We propose that this 54 approach could equip upper-limb neuroprostheses with direct access to texture features 55 acquired during active exploration of natural objects. 57Sensory neuroprostheses offer the promise of restoring perceptual function to people 59 with impaired sensation 1,2 . In such devices, diminished sensory modalities (e.g., hearing 3 , 60 vision 4,5 , or cutaneous touch 6-8 ) are reenacted through streams of artificial input to the nervous 61 system, typically using electrical stimulation of nerve fibers in the periphery or neurons in the 62 central nervous system. Restored cutaneous touch, in particular, would be of great benefit for 63 the users of upper-limb prostheses, who place a high priority on the ability to perform functions 64 without the necessity to constantly engage visual attention 9 . This could be achieved through the 65 addition of artificial somatosensory channels to the prosthetic device 1 . Such an approach would 66 endow persons suffering from limb loss 10-12 , paralysis 1,13 or somatosensory deficits with the 67 ability to perform active tactile exploration of their physical environment and aid in dexterous 68 object manipulation [14][15][16][17] . 69Previously we demonstrated that motor and sensory functions could be simultaneously 70 enacted though a bidirectional neuroprosthetic system, called a brain-machine-brain interface 71 (BMBI) 18 . In that demonstration, the active exploration enabled by our BMBI-driven 72 neuroprosthesis used a limited and fixed set of ICMS temporal patterns to generate artificial 73 sensory inputs that mimicked the sense of flutter-vibration. However, it r...

show abstract

Embedding a Panoramic Representation of Infrared Light in the Adult Rat Somatosensory Cortex through a Sensory Neuroprosthesis

Cited by 28 publications

References 58 publications

Learning active sensing strategies using a sensory brain–machine interface

Learning active sensing strategies using a sensory brain–machine interface

A Mathematical Theory of Cortex-Receptor Artificial Extension

Creating a neuroprosthesis for active tactile exploration of textures

Contact Info

Product

Resources

About