Spike Camera and Its Coding Methods

Dong, Siwei; Huang, Tiejun

doi:10.1109/dcc.2017.69

Cited by 56 publications

(23 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…One of the potential applications of the balanced network's fast response property is to process Spike Camera data in real time. Spike Camera is a newly developed neuromorphic hardware that encodes visual signals with spikes (Dong et al, 2017 ). It consists of artificial ganglion cells, each corresponding to a pixel, that linearly integrate the luminance intensity and fire a spike upon reaching the threshold, converting continuous visual information to discrete spikes.…”

Section: Simulation Resultsmentioning

confidence: 99%

“…Meanwhile, many artificial engineering systems have high demands for real-time processing of rapidly varying signals. This is exemplified by the recently developed Spike Camera (Dong et al, 2017 ), which has a sampling rate of up to 40, 000 frames per second (fps), far surpassing conventional cameras' 60 fps. This allows it to capture high-speed objects and their textual details, which can be used on real-time motion detection, tracking, and recognition if we have the appropriate algorithms and computing platforms.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Excitation-Inhibition Balanced Neural Networks for Fast Signal Detection

Tian

Huang

et al. 2020

Front. Comput. Neurosci.

Self Cite

View full text Add to dashboard Cite

Excitation-inhibition (E-I) balanced neural networks are a classic model for modeling neural activities and functions in the cortex. The present study investigates the potential application of E-I balanced neural networks for fast signal detection in brain-inspired computation. We first theoretically analyze the response property of an E-I balanced network, and find that the asynchronous firing state of the network generates an optimal noise structure enabling the network to track input changes rapidly. We then extend the homogeneous connectivity of an E-I balanced neural network to include local neuronal connections, so that the network can still achieve fast response and meanwhile maintain spatial information in the face of spatially heterogeneous signal. Finally, we carry out simulations to demonstrate that our model works well.

show abstract

Section: Simulation Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Excitation-Inhibition Balanced Neural Networks for Fast Signal Detection

Tian

Huang

et al. 2020

Front. Comput. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Embedding the encoder/decoder into the retinal prosthesis has been suggested as a promising direction for visual restoration [54]. Such an approach of studying spike encoding and decoding of visual scenes with neural spikes will be crucial for the next generation of neuromorphic computing, including artificial visual system [53], where the data format processed on chips are digital spikes [55]. One expects that the close interaction of the algorithms based on spike data and CNN [56], [57], [58] with neuromorphic chips [59], [60] will greatly expand our computing capacity.…”

Section: Summary and Discussionmentioning

confidence: 99%

Revealing Fine Structures of the Retinal Receptive Field by Deep-Learning Networks

Yan

Zheng

Jia

et al. 2022

IEEE Trans. Cybern.

Self Cite

View full text Add to dashboard Cite

Deep convolutional neural networks (CNNs) have demonstrated impressive performance on many visual tasks. Recently, they became useful models for the visual system in neuroscience. However, it is still not clear what are learned by CNNs in terms of neuronal circuits. When a deep CNN with many layers is used for the visual system, it is not easy to compare the structure components of CNN with possible neuroscience underpinnings due to highly complex circuits from the retina to higher visual cortex. Here we address this issue by focusing on single retinal ganglion cells with biophysical models and recording data from animals. By training CNNs with white noise images to predict neuronal responses, we found that fine structures of the retinal receptive field can be revealed. Specifically, convolutional filters learned are resembling biological components of the retinal circuit. This suggests that a CNN learning from one single retinal cell reveals a minimal neural network carried out in this cell. Furthermore, when CNNs learned from different cells are transferred between cells, there is a diversity of transfer learning performance, which indicates that CNNs are cell-specific. Moreover, when CNNs are transferred between different types of input images, here white noise v.s. natural images, transfer learning shows a good performance, which implies that CNN indeed captures the full computational ability of a single retinal cell for different inputs. Taken together, these results suggest that CNN could be used to reveal structure components of neuronal circuits, and provide a powerful model for neural system identification.

show abstract

“…The main feature of these algorithms is to make use of neural spikes. Advancements of recent artificial intelligence computing align with the development of the next generation of neuromorphic chips and devices, where the new data format is processed as spikes or events [186,187,188,189,190]. Therefore, the methods can be applied for neuromorphic visual cameras with spike or event signals as well.…”

Section: Discussionmentioning

confidence: 99%

Toward the Next Generation of Retinal Neuroprosthesis: Visual Computation with Spikes

Liu

Jia

et al. 2020

Engineering

Self Cite

View full text Add to dashboard Cite

Neuroprosthesis, as one type of precision medicine device, is aiming for manipulating neuronal signals of the brain in a closed-loop fashion, together with receiving stimulus from the environment and controlling some part of our brain/body. In terms of vision, incoming information can be processed by the brain in millisecond interval. The retina computes visual scenes and then sends its output as neuronal spikes to the cortex for further computation. Therefore, the neuronal signal of interest for retinal neuroprosthesis is spike. Closed-loop computation in neuroprosthesis includes two stages: encoding stimulus to neuronal signal, and decoding it into stimulus. Here we review some of the recent progress about visual computation models that use spikes for analyzing natural scenes, including static images and dynamic movies. We hypothesize that for a better understanding of computational principles in the retina, one needs a hypercircuit view of the retina, in which different functional network motifs revealed in the cortex neuronal network should be taken into consideration for the retina. Different building blocks of the retina, including a diversity of cell types and synaptic connections, either chemical synapses or electrical synapses (gap junctions), make the retina an ideal neuronal network to adapt the computational techniques developed in artificial intelligence for modeling of encoding/decoding visual scenes. Altogether, one needs a systems approach of visual computation with spikes to advance the next generation of retinal neuroprosthesis as an artificial visual system.

show abstract

Spike Camera and Its Coding Methods

Cited by 56 publications

References 9 publications

Excitation-Inhibition Balanced Neural Networks for Fast Signal Detection

Excitation-Inhibition Balanced Neural Networks for Fast Signal Detection

Revealing Fine Structures of the Retinal Receptive Field by Deep-Learning Networks

Toward the Next Generation of Retinal Neuroprosthesis: Visual Computation with Spikes

Contact Info

Product

Resources

About