Brain-inspired models for visual object recognition: an overview

Yang, Xi; Yan, Jie; Wang, Wen; Li, Shaoyi; Hu, Bo; Lin, Jian Hung

doi:10.1007/s10462-021-10130-z

Cited by 20 publications

(14 citation statements)

References 183 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To achieve a more human-like behavior in artificial neural networks, it is necessary to increase layers and neurons in the networks, thereby enhancing their transfer learning ability. Figure 1a illustrates a schematic diagram of pre-learning method and cross-regional hierarchical structure in biological brain 37 – 40 . The artificial neural network realized by imitating this structure shows great ability after extensive basic learning (pre-learning) and achieves high accuracy in untrained tasks, which is very similar to human learning ability.…”

Section: Resultsmentioning

confidence: 99%

Cross-layer transmission realized by light-emitting memristor for constructing ultra-deep neural network with transfer learning ability

Chen,

Lin,

Yang

et al. 2024

Nat Commun

View full text Add to dashboard Cite

Deep neural networks have revolutionized several domains, including autonomous driving, cancer detection, and drug design, and are the foundation for massive artificial intelligence models. However, hardware neural network reports still mainly focus on shallow networks (2 to 5 layers). Implementing deep neural networks in hardware is challenging due to the layer-by-layer structure, resulting in long training times, signal interference, and low accuracy due to gradient explosion/vanishing. Here, we utilize negative ultraviolet photoconductive light-emitting memristors with intrinsic parallelism and hardware-software co-design to achieve electrical information’s optical cross-layer transmission. We propose a hybrid ultra-deep photoelectric neural network and an ultra-deep super-resolution reconstruction neural network using light-emitting memristors and cross-layer block, expanding the networks to 54 and 135 layers, respectively. Further, two networks enable transfer learning, approaching or surpassing software-designed networks in multi-dataset recognition and high-resolution restoration tasks. These proposed strategies show great potential for high-precision multifunctional hardware neural networks and edge artificial intelligence.

show abstract

Section: Resultsmentioning

confidence: 99%

Cross-layer transmission realized by light-emitting memristor for constructing ultra-deep neural network with transfer learning ability

Chen,

Lin,

Yang

et al. 2024

Nat Commun

View full text Add to dashboard Cite

show abstract

“…All participants yielded qualitatively similar results ( Supplementary Figures 9-10 ). This proves that our EEG dataset allows for the successful training of DNNs in an end-to-end fashion, paving the way for a stronger symbiosis between brain data and deep learning models benefitting both neuroscientists interested in building better models of the brain (Seeliger et al, 2021; Khosla et al, 2021; Allen et al, 2021) and computer scientists interested in creating better performing and more brain-like artificial intelligence algorithms through inductive biases from biological intelligence (Sinz et al, 2019; Hassabis et al, 2017; Ullman, 2019; Toneva & Wehbe, 2019; Yang et al, 2022).…”

Section: Resultsmentioning

confidence: 99%

“…The end-toend approach opens the doors to training complex computational algorithms directly with brain data, potentially leading to models which more closely mimic the internal representation of the visual system (Sinz et al, 2019;Allen et al, 2021). This will in turn make it possible for computer scientists to use the neural representations of biological systems as inductive biases to improve artificial systems under the assumption that increasing the brain-likeness of computer models could increase their performance in tasks at which humans excel (Sinz et al, 2019;Hassabis et al, 2017;Ullman, 2019;Toneva & Wehbe, 2019;Yang et al, 2022).…”

Section: End-to-end Encodingmentioning

confidence: 99%

“…We release the dataset as a tool to foster research in computational neuroscience and to bridge the gap between biological and artificial vision. We believe this will be of great use to further understanding of visual object recognition through the development of high-temporal resolution computational models of the visual brain, and to optimize artificial intelligence models through biological intelligence data (Sinz et al, 2019; Hassabis et al, 2017; Ullman, 2019; Toneva & Wehbe, 2019; Yang et al, 2022). Also all code used to generate the presented results accompanies the data release.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A large and rich EEG dataset for modeling human visual object recognition

Gifford

Dwivedi

Roig

et al. 2022

Preprint

View full text Add to dashboard Cite

The human brain achieves visual object recognition through multiple stages of nonlinear transformations operating at a millisecond scale. To predict and explain these rapid transformations, computational neuroscientists employ machine learning modeling techniques. However, state-of-the-art models require massive amounts of data to properly train, and to the present day there is a lack of vast brain datasets which extensively sample the temporal dynamics of visual object recognition. Here we collected a large and rich dataset of high temporal resolution EEG responses to images of objects on a natural background. This dataset includes 10 participants, each with 82,160 trials spanning 16,740 image conditions. Through computational modeling we established the quality of this dataset in five ways. First, we trained linearizing encoding models that successfully synthesized the EEG responses to arbitrary images. Second, we correctly identified the recorded EEG data image conditions in a zero-shot fashion, using EEG synthesized responses to hundreds of thousands of candidate image conditions. Third, we show that both the high number of conditions as well as the trial repetitions of the EEG dataset contribute to the trained models' prediction accuracy. Fourth, we built encoding models whose predictions well generalize to novel participants. Fifth, we demonstrate full end-to-end training of randomly initialized DNNs that output M/EEG responses for arbitrary input images. We release this dataset as a tool to foster research in visual neuroscience and computer vision.

show abstract

“…It is assumed that the primate visual system is organized into two separate processing pathways in the visual cortex, namely, the dorsal pathway and the ventral pathway. While the dorsal pathway is responsible for the spatial recognition of objects as well as actions and manipulations such as grasping, the ventral pathway is responsible for recognizing the type of object based on its form or motion [52]. Bonner et al [6] recently showed that the sensory coding of objects in the ventral cortex of the human brain is related to statistical embeddings of object or word co-occurrences.…”

Section: Preliminariesmentioning

confidence: 99%

Context-Driven Visual Object Recognition Based on Knowledge Graphs

Monka

Halilaj

Rettinger

2022

The Semantic Web – ISWC 2022

View full text Add to dashboard Cite

Current deep learning methods for object recognition are purely data-driven and require a large number of training samples to achieve good results. Due to their sole dependence on image data, these methods tend to fail when confronted with new environments where even small deviations occur. Human perception, however, has proven to be significantly more robust to such distribution shifts. It is assumed that their ability to deal with unknown scenarios is based on extensive incorporation of contextual knowledge. Context can be based either on object co-occurrences in a scene or on memory of experience. In accordance with the human visual cortex which uses context to form different object representations for a seen image, we propose an approach that enhances deep learning methods by using external contextual knowledge encoded in a knowledge graph. Therefore, we extract different contextual views from a generic knowledge graph, transform the views into vector space and infuse it into a DNN. We conduct a series of experiments to investigate the impact of different contextual views on the learned object representations for the same image dataset. The experimental results provide evidence that the contextual views influence the image representations in the DNN differently and therefore lead to different predictions for the same images. We also show that context helps to strengthen the robustness of object recognition models for out-of-distribution images, usually occurring in transfer learning tasks or real-world scenarios.

show abstract

Brain-inspired models for visual object recognition: an overview

Cited by 20 publications

References 183 publications

Cross-layer transmission realized by light-emitting memristor for constructing ultra-deep neural network with transfer learning ability

Cross-layer transmission realized by light-emitting memristor for constructing ultra-deep neural network with transfer learning ability

A large and rich EEG dataset for modeling human visual object recognition

Context-Driven Visual Object Recognition Based on Knowledge Graphs

Contact Info

Product

Resources

About