What can computer vision learn from visual neuroscience? Introduction to the special issue

Chen, Kexin; Kashyap, Hirak J.; Krichmar, Jeffrey L.; Li, Xiumin

doi:10.1007/s00422-023-00977-6

Cited by 1 publication

(1 citation statement)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, foundation models based on deep artificial neural networks have shown robust representations of their modeling domain and achieved breakthroughs for accurately predicting neuronal responses to arbitrary natural images in a visual cortex [1][2][3][4][5][6]. However, despite the appearance of deep learning artificial neural networks, which have recently shown remarkable capability on a broad range of computational tasks, these models require high energy consumption and need to run on graphics processors that consume many kilowatts of power [7]. Therefore, brain-inspired algorithmic models and neuromorphic hardware processors are increasingly emerging and can potentially lead to low-power intelligent systems for large-scale real-world applications.…”

Section: Introductionmentioning

confidence: 99%

A Visually Inspired Computational Model for Recognition of Optic Flow

Li,

Lin,

et al. 2023

Mathematics

Self Cite

View full text Add to dashboard Cite

Foundation models trained on vast quantities of data have demonstrated impressive performance in capturing complex nonlinear relationships and accurately predicting neuronal responses. Due to the fact that deep learning neural networks depend on massive amounts of data samples and high energy consumption, foundation models based on spiking neural networks (SNNs) have the potential to significantly reduce calculation costs by training on neuromorphic hardware. In this paper, a visually inspired computational model composed of an SNN and echo state network (ESN) is proposed for the recognition of optic flow. The visually inspired SNN model serves as a foundation model that is trained using spike-timing-dependent plasticity (STDP) for extracting core features. The ESN model makes readout decisions for recognition tasks using the linear regression method. The results show that STDP can perform similar functions as non-negative matrix decomposition (NMF), i.e., generating sparse and linear superimposed readouts based on basis flow fields. Once the foundation model is fully trained from enough input samples, it can considerably reduce the training samples required for ESN readout learning. Our proposed SNN-based foundation model facilitates efficient and cost-effective task learning and could also be adapted to new stimuli that are not included in the training of the foundation model. Moreover, compared with the NMF algorithm, the foundation model trained using STDP does not need to be retrained during the testing procedure, contributing to a more efficient computational performance.

show abstract