EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients

Bendou, Yassir; Hu, Yuqing; Lafargue, Raphael; Lioi, Giulia; Pasdeloup, Bastien; Pateux, Stéphane; Gripon, Vincent

doi:10.48550/arxiv.2201.09699

Cited by 6 publications

(18 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We found ViT performed the best accuracy for both 1-shot and 5-shot learning tasks. Note that other one-shot learning methods in previous study can obtain the same degree of accuracy without using such an excellent backbone DNN like ViT (e.g., 82% [40] and 87% [41] with ResNet-12), and thus DONE is not the best method for accuracy. However, these results suggest that DONE with ViT is already at a level of practical uses.…”

Section: Evaluation Of Dnns In Donementioning

confidence: 86%

“…Metric learning such as using Siamese network [13] is useful for tasks that require one-shot learning, e.g., face recognition. A Data-augmentation approach generatively increases the number of training inputs [41,18,24]. This approach includes various types such as semi-supervised approaches and example generation using Generative Adversarial Networks [10].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

It's DONE: Direct ONE-shot learning without training optimization

Hosoda¹,

Nishida²,

Seno³

et al. 2022

Preprint

View full text Add to dashboard Cite

Learning a new concept from one example is a superior function of human brain and it is drawing attention in the field of machine learning as one-shot learning task. In this paper, we propose the simplest method for this task, named Direct ONE-shot learning (DONE). DONE adds a new class to a pretrained deep neural network (DNN) classifier with neither training optimization nor other-classes modification. DONE is inspired by Hebbian theory and directly uses the neural activity input of the final dense layer obtained from a data that belongs to the new additional class as the connectivity weight (synaptic strength) with a newly-provided-output neuron for the new class. DONE requires just one inference for obtaining the output of the final dense layer and its procedure is simple, deterministic, not requiring parameter tuning and hyperparameters. The performance of DONE depends entirely on the pretrained DNN model used as a backbone model, and we confirmed that DONE with a well-trained backbone model performs a practical-level accuracy. DONE has some advantages including a DNN's practical use that is difficult to spend high cost for a training, an evaluation of existing DNN models, and the understanding of the brain. DONE might be telling us one-shot learning is an easy task that can be achieved by a simple principle not only for humans but also for current well-trained DNN models.

show abstract

Section: Evaluation Of Dnns In Donementioning

confidence: 86%

Section: Related Workmentioning

confidence: 99%

It's DONE: Direct ONE-shot learning without training optimization

Hosoda¹,

Nishida²,

Seno³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Finally, we conduct comparison of our performance with the stateof-the-art results on three benchmarks: miniImageNet, tieredIm-ageNet, and CUB. We choose the state-of-the-art FSL methods ProtoNet [36], FEAT [47], and ASY [3] as the first group of models.…”

Section: Comparison With the State-of-the-art Methodsmentioning

confidence: 99%

Few-shot Open-set Recognition Using Background as Unknowns

Song¹,

Zhang²,

Lin³

2022

Preprint

View full text Add to dashboard Cite

Bkg feat Bkg feat a) Close-set classification b) Few-shot open-set classificationFigure 1: a) Close-set classification. Standard classification classifies all inputs to a set of pre-defined training classes. b) Fewshot open-set classification task aims to classify images from seen and unseen classes given only limited training samples of seen classes. We propose to utilize the background regions at the training phase as the pseudo unseen classes to train the classifier that can directly classify seen and unseen images.

show abstract

“…There are two main frameworks in the field of FSC: 1) only one unlabelled sample is processed at a time for class predictions, which is called inductive FSC, and 2) the entire unlabelled samples are available for further estimations, which is called transductive FSC. Inductive methods focus on training a feature extractor that generalizes well the embedding in a feature sub-space, they include meta learning methods such as [12,26,2,40,30,37] that train a model in an episodic manner, and transfer learning methods [8,28,48,5,3,33] that train a model with a set of mini-batches.…”

Section: Related Workmentioning

confidence: 99%

“…As for the hyper-parameters, we set T km = 10, T vb = 50, s max = 2 for the balanced setting; T km = 50, T vb = 50, s max = 1 for the unbalanced setting, and we use the same VB priors for all settings. To further show the functionality of our proposed method on different backbones and other benchmarks, we tested BAVARDAGE on a recent high performing feature extractor trained on a ResNet-12 (RN12) neural model [28,3], and we report the accuracy in Table 1 and in Appendix with various settings.…”

Section: Implementation Detailsmentioning

confidence: 99%

Adaptive Dimension Reduction and Variational Inference for Transductive Few-Shot Classification

Hu¹,

Pateux²,

Gripon³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Transductive Few-Shot learning has gained increased attention nowadays considering the cost of data annotations along with the increased accuracy provided by unlabelled samples in the domain of few shot. Especially in Few-Shot Classification (FSC), recent works explore the feature distributions aiming at maximizing likelihoods or posteriors with respect to the unknown parameters. Following this vein, and considering the parallel between FSC and clustering, we seek for better taking into account the uncertainty in estimation due to lack of data, as well as better statistical properties of the clusters associated with each class. Therefore in this paper we propose a new clustering method based on Variational Bayesian inference, further improved by Adaptive Dimension Reduction based on Probabilistic Linear Discriminant Analysis. Our proposed method significantly improves accuracy in the realistic unbalanced transductive setting on various Few-Shot benchmarks when applied to features used in previous studies, with a gain of up to 6% in accuracy. In addition, when applied to balanced setting, we obtain very competitive results without making use of the class-balance artefact which is disputable for practical use cases. We also provide the performance of our method on a high performing pretrained backbone, with the reported results further surpassing the current state-of-the-art accuracy, suggesting the genericity of the proposed method.

show abstract

EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients

Cited by 6 publications

References 27 publications

It's DONE: Direct ONE-shot learning without training optimization

It's DONE: Direct ONE-shot learning without training optimization

Few-shot Open-set Recognition Using Background as Unknowns

Adaptive Dimension Reduction and Variational Inference for Transductive Few-Shot Classification

Contact Info

Product

Resources

About