Prototype Rectification for Few-Shot Learning

Liu, Jinlu; Song, Liang; Qin, Yongqiang

doi:10.1007/978-3-030-58452-8_43

Cited by 186 publications

(98 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[5] proposed transductive fine-tuning, which pursues outputs with a peaked posterior or low Shannon entropy, and a hardness metric to deliver a standardized evaluation protocol. [26] proposed the prototype rectification, which lowers the class prototype's intra-class bias and cross-class bias and verifies the method theoretically. A synthetic information bottleneck (SIB) [12] introduced an empirical Bayes approach and a two-network architecture consisting of a synthetic gradient network and an initialization network to perform the synthetic gradient descent.…”

Section: Related Workmentioning

confidence: 69%

“…We mainly investigated the alternating direction method (ADM) version of the TIM algorithm, which is faster than the gradient descent (GD) version 2 . We have added a prototype estimation technique [26], [59] to TIM. This further improved the 1-shot classification accuracy.…”

Section: Methodsmentioning

confidence: 99%

“…Transductive few-shot methods [1], [5], [12], [26], [31], [50], [59] assume that the model simultaneously accesses all the query set. A transductive episodic-wise adaptive metric (TEAM) [31] defined the optimization process as a standard semi-definite programming problem to train a generalizable classifier.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Augmenting Few-Shot Learning With Supervised Contrastive Learning

Lee

Yoo

2021

IEEE Access

View full text Add to dashboard Cite

Few-shot learning deals with a small amount of data which incurs insufficient performance with conventional cross-entropy loss. We propose a pretraining approach for few-shot learning scenarios. That is, considering that the feature extractor quality is a critical factor in few-shot learning, we augment the feature extractor using a contrastive learning technique. It is reported that supervised contrastive learning applied to base class training in transductive few-shot training pipeline leads to improved results, outperforming the state-of-the-art methods on Mini-ImageNet and CUB. Furthermore, our experiment shows that a much larger dataset is needed to retain few-shot classification accuracy when domain-shift degradation exists, and if our method is applied, the need for a large dataset is eliminated. The accuracy gain can be translated to a runtime reduction of 3.87× in a resource-constrained environment.

show abstract

Section: Related Workmentioning

confidence: 69%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Augmenting Few-Shot Learning With Supervised Contrastive Learning

Lee

Yoo

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…We can divide meta‐learning methods into three categories 140 : Metric learning methods (i.e., MatchingNets, 113 ProtoNets, 114 RelationNets, 115 Graph neural network (GraphNN), 116 Ridge regression, 117 TransductiveProp, 118 Fine‐tuning Baseline, 119 URT, 120 DSN‐MR, 121 CDFS, 122 DeepEMD, 123 EPNet, 124 ACC + Amphibian, 125 FEAT, 126 MsSoSN + SS + SD + DD, 127 RFS, 128 RFS + CRAT, 129 IDA, 130 LR + ICI, 131 FEAT + MLMT, 132 BOHB, 133 CSPN, 134 SUR, 135 SKD, 136 TAFSSL, 137 TRPN, 138 and TransMatch 139 ) learn a similarity space in which learning is particularly efficient for few‐shot examples. Memory network methods (i.e., Meta Networks, 103 TADAM, 104 MCFS, 105 and MRN 106 ) learn to store “experience” when learning seen tasks and then generalize it to unseen tasks. …”

Section: Methodsmentioning

confidence: 99%

“…MsSoSN + SS + SD + DD, 127 RFS, 128 RFS + CRAT, 129 IDA, 130 LR + ICI, 131 FEAT + MLMT, 132 BOHB, 133 CSPN, 134 SUR, 135 SKD, 136 TAFSSL, 137 TRPN, 138 and TransMatch 139 ) learn a similarity space in which learning is particularly efficient for few-shot examples. 2.…”

mentioning

confidence: 99%

Fighting fire with fire: A spatial–frequency ensemble relation network with generative adversarial learning for adversarial image classification

et al. 2021

View full text Add to dashboard Cite

Adversarial images generated by generative adversarial networks are not close to any existing benign images, and contain nonrobust features that have been identified as critical to the robustness of a machine learning model. Since adversarial images have an underlying distribution that differs from normal images, these kinds of images can offer valuable features for training a robust model. To deal with these special features, we focus on a novel machine learning task of adversarial images classification, where adversarial images can be used to investigate the problem of classifying adversarial images themselves. In the setting of this novel task, adversarial images are the ONLY kind of data used in training and testing, rather than not just a set of testing images as usual. To this end, we propose a novel spatial–frequency ensemble relation network with generative adversarial learning. First, we present a spatial–frequency ensemble representation learning to extract the feature of training images. Second, we design a meta‐learning‐based relation model to gain the relationship between images. Third, to achieve a robust model, we utilize generative adversarial learning and transform the relationship into a Jacobian matrix. Finally, we design a discriminator model that determines whether an adversarial image is from the matching category or not. Experimental results demonstrate that our approach achieves significantly higher performance compared with other state‐of‐the‐arts.

show abstract

Memristive Cosine‐Similarity‐Based Few‐Shot Learning with Lifelong Memory Adaptation

Zhou

Ling

Bao

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

Deep learning in recent years has drawn much attention for its outperformance in the fields of computer vision, [1] speech recognition, [2] and natural language processing. [3] The implementation of these tasks relies heavily on the use of deep neural networks (DNNs) to model the prior knowledge distribution in massive amounts of data. In most cases, acquiring training data for DNNs is much more important than the DNN models. [4] Nevertheless, processing data with a limited size of training materials is more in line with the human brain, such as a kid can easily distinguish a dog from cats once he is taught what a dog looks like with one sample. But this problem faces great challenges for the mainstream neural networks because DNNs fail to learn the features of a dog in only one picture. Few-shot learning (FSL), a branch of meta-learning, [5] has been accordingly proposed to build reliable machine learning models using a few labeled examples, just like the way humans work. Among the popular FSL algorithms, the use of dynamic external memory combined with neural network control, [6] also known as memory-augmented neural network (MANN), [7] stands out for the outperformance on FSL tasks. The external memory, usually DRAM, [7,8] is used to remember the features of the insufficient samples for a long time and assists in improving the performance of FSL consistently. However, the time delay caused by the data communication between the memory and the running neural network is costly and degrades the performance of data search using external memory. The introduction of emerging non-volatile memory has become a viable solution to this problem due to its non-volatile nature, which greatly improves search speed and shows the potential for lifelong learning tasks.Various non-volatile devices, such as the flash, [9] FeFETs, [10,11] resistive random-access memory (RRAM), [12][13][14] and phase-change memory (PCM), [15] have been used as external memory to accelerate the similarity computation in MANN. Ni et al. [11] proposed a two-FeFET scheme as a ternary content addressable memory (ternary CAM) cell to calculate the Hamming distances of binary vectors for data indexing. However, two problems arising from this framework are the reduced accuracy compared to digital computers and the long encoding bits for the binarized vectors. Then Karunaratne et al. [15] designed a new attention mechanism to improve the accuracy of similarity-based FSL with the cosine distance of binarized vectors on PCMs. The length of the embedding vectors in refs. [11,15] are 512 bits with the Omniglot dataset, leading to a large area and power overhead of external memory. Kazemi et al. [10] developed an analog CAM with FeFET using the structure in ref. [11] and tried to reduce the encoding

show abstract

Prototype Rectification for Few-Shot Learning

Cited by 186 publications

References 13 publications

Augmenting Few-Shot Learning With Supervised Contrastive Learning

Augmenting Few-Shot Learning With Supervised Contrastive Learning

Fighting fire with fire: A spatial–frequency ensemble relation network with generative adversarial learning for adversarial image classification

Memristive Cosine‐Similarity‐Based Few‐Shot Learning with Lifelong Memory Adaptation

Contact Info

Product

Resources

About