Few Shot Model based on Weight Imprinting with Multiple Projection Head

Cristovao, Paulino; Nakada, Hidemoto; Tanimura, Yusuke; Asoh, Hideki

doi:10.1109/imcom53663.2022.9721726

Cited by 2 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, Qi's method does not address the statistical discrepancies between x and w j , which limits the range of applicable backbone DNNs (for more details, see Section 4). While various studies have developed Qi's method further, making it more complex and adaptable (Dhillon et al, 2019 ; Li et al, 2021 ; Passalis et al, 2021 ; Cristovao et al, 2022 ; Zhu et al, 2022 ), to the best of our knowledge, none have sought to simplify it or effectively tackle the transformation problem from x to w j .…”

Section: Weight Imprintings and Hebbian Interpretationmentioning

confidence: 99%

A single fast Hebbian-like process enabling one-shot class addition in deep neural networks without backbone modification

Hosoda,

Nishida,

Seno

et al. 2024

Front. Neurosci.

View full text Add to dashboard Cite

One-shot learning, the ability to learn a new concept from a single instance, is a distinctive brain function that has garnered substantial interest in machine learning. While modeling physiological mechanisms poses challenges, advancements in artificial neural networks have led to performances in specific tasks that rival human capabilities. Proposing one-shot learning methods with these advancements, especially those involving simple mechanisms, not only enhance technological development but also contribute to neuroscience by proposing functionally valid hypotheses. Among the simplest methods for one-shot class addition with deep learning image classifiers is “weight imprinting,” which uses neural activity from a new class image data as the corresponding new synaptic weights. Despite its simplicity, its relevance to neuroscience is ambiguous, and it often interferes with original image classification, which is a significant drawback in practical applications. This study introduces a novel interpretation where a part of the weight imprinting process aligns with the Hebbian rule. We show that a single Hebbian-like process enables pre-trained deep learning image classifiers to perform one-shot class addition without any modification to the original classifier's backbone. Using non-parametric normalization to mimic brain's fast Hebbian plasticity significantly reduces the interference observed in previous methods. Our method is one of the simplest and most practical for one-shot class addition tasks, and its reliance on a single fast Hebbian-like process contributes valuable insights to neuroscience hypotheses.

show abstract

Section: Weight Imprintings and Hebbian Interpretationmentioning

confidence: 99%

A single fast Hebbian-like process enabling one-shot class addition in deep neural networks without backbone modification

Hosoda,

Nishida,

Seno

et al. 2024

Front. Neurosci.

View full text Add to dashboard Cite

show abstract

“…Furthermore, Qi's method does not address the statistical discrepancies between x and w j , which limits the range of applicable backbone DNNs (for more details, see Section 4). While various studies have developed Qi's method further, making it more complex and adaptable (Dhillon et al, 2019;Li et al, 2021;Passalis et al, 2021;Cristovao et al, 2022;Zhu et al, 2022), to the best of our knowledge, none have sought to simplify it or effectively tackle the transformation problem from x to w j .…”

Section: Weight Imprintings and Hebbian Interpretation Qi's Weight I...mentioning

confidence: 99%

Data_Sheet_1.PDF

View full text Add to dashboard Cite

Few Shot Model based on Weight Imprinting with Multiple Projection Head

Cited by 2 publications

References 13 publications

A single fast Hebbian-like process enabling one-shot class addition in deep neural networks without backbone modification

A single fast Hebbian-like process enabling one-shot class addition in deep neural networks without backbone modification

Data_Sheet_1.PDF

Contact Info

Product

Resources

About