Hebbian semi-supervised learning in a sample efficiency setting

Lagani, Gabriele; Falchi, Fabrizio; Gennaro, Claudio; Amato, Giuseppe

doi:10.1016/j.neunet.2021.08.003

Cited by 18 publications

(17 citation statements)

References 319 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The details of Hebbian learning theories are outside the scope of this paper, although a plethora of Hebbian-based learning algorithms have been proposed. We refer the interested reader to [1,13,[24][25][26][27]. In particular, in our previous works [24,25], we have highlighted the merits of the nonlinear Hebbian Principal Component Analysis (HPCA).…”

Section: Unsupervised Hebbian Pre-trainingmentioning

confidence: 99%

“…We refer the interested reader to [1,13,[24][25][26][27]. In particular, in our previous works [24,25], we have highlighted the merits of the nonlinear Hebbian Principal Component Analysis (HPCA). This is derived by minimizing the representation error:…”

Section: Unsupervised Hebbian Pre-trainingmentioning

confidence: 99%

“…Where 𝜂 is the learning rate. Since we found this learning rule to be particularly effective in semi-supervised regimes with label scarcity [24,25], this is the rule that we adopted in the unsupervised pre-training phase of our method.…”

Section: Unsupervised Hebbian Pre-trainingmentioning

confidence: 99%

“…Unsupervised pre-training is known to be beneficial before further fine-tuning stage [7,19,29,38]. Concerning the Hebbian pre-training phase, a nonlinear learning rule for Hebbian Principal Component Analysis (HPCA) was considered [24,25,27]. For the supervised fine-tuning stage, we considered sample efficiency scenarios, in which only a fraction of the training data is assumed to be labeled.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Deep Features for CBIR with Scarce Data using Hebbian Learning

Lagani¹,

Bacciu²,

Gallicchio³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Features extracted from Deep Neural Networks (DNNs) have proven to be very effective in the context of Content Based Image Retrieval (CBIR). In recent work, biologically inspired Hebbian learning algorithms have shown promises for DNN training. In this contribution, we study the performance of such algorithms in the development of feature extractors for CBIR tasks. Specifically, we consider a semi-supervised learning strategy in two steps: first, an unsupervised pre-training stage is performed using Hebbian learning on the image dataset; second, the network is fine-tuned using supervised Stochastic Gradient Descent (SGD) training. For the unsupervised pre-training stage, we explore the nonlinear Hebbian Principal Component Analysis (HPCA) learning rule. For the supervised finetuning stage, we assume sample efficiency scenarios, in which the amount of labeled samples is just a small fraction of the whole dataset. Our experimental analysis, conducted on the CIFAR10 and CIFAR100 datasets, shows that, when few labeled samples are available, our Hebbian approach provides relevant improvements compared to various alternative methods.

show abstract

Section: Unsupervised Hebbian Pre-trainingmentioning

confidence: 99%

Section: Unsupervised Hebbian Pre-trainingmentioning

confidence: 99%

Section: Unsupervised Hebbian Pre-trainingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep Features for CBIR with Scarce Data using Hebbian Learning

Lagani¹,

Bacciu²,

Gallicchio³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [ 24 ], BCM theory, Competitive Hebbian Learning, and Stochastic Gradient Descent are considered to derive a new learning rule. The integration of Hebbian-based learning with ConvNets has also been proposed [ 25 , 26 , 27 , 28 ], but BCM learning rules have been barely considered [ 29 ]. In addition, some of the previous works focused on improving the TDL algorithm, taking into account the results of [ 1 ], which includes the articles by [ 30 , 31 , 32 , 33 ].…”

Section: Related Workmentioning

confidence: 99%

A Hebbian Approach to Non-Spatial Prelinguistic Reasoning

Canto

Calvo

2022

Brain Sciences

View full text Add to dashboard Cite

This research integrates key concepts of Computational Neuroscience, including the Bienestock-CooperMunro (BCM) rule, Spike Timing-Dependent Plasticity Rules (STDP), and the Temporal Difference Learning algorithm, with an important structure of Deep Learning (Convolutional Networks) to create an architecture with the potential of replicating observations of some cognitive experiments (particularly, those that provided some basis for sequential reasoning) while sharing the advantages already achieved by the previous proposals. In particular, we present Ring Model B, which is capable of associating visual with auditory stimulus, performing sequential predictions, and predicting reward from experience. Despite its simplicity, we considered such abilities to be a first step towards the formulation of more general models of prelinguistic reasoning.

show abstract