Few-shot learning with adaptively initialized task optimizer: a practical meta-learning approach

Abstract

Considering the data collection and labeling cost in real-world applications, training a model with limited examples is an essential problem in machine learning, visual recognition, etc. Directly training a model on such few-shot learning (FSL) tasks falls into the over-fitting dilemma, which would turn to an effective task-level inductive bias as a key supervision. By treating the few-shot task as an entirety, extracting task-level pattern, and learning a task-agnostic model initialization, the model-agnostic meta-learning (MAML) framework enables the applications of various models on the FSL tasks. Given a training set with a few examples, MAML optimizes a model via fixed gradient descent steps from an initial point chosen beforehand. Although this general framework possesses empirically satisfactory results, its initialization neglects the task-specific characteristics and aggravates the computational burden as well. In this manuscript, we propose our AdaptiVely InitiAlized Task OptimizeR (Aviator) approach for few-shot learning, which incorporates task context into the determination of the model initialization. This task-specific initialization facilitates the model optimization process so that it obtains high-quality model solutions efficiently. To this end, we decouple the model and apply a set transformation over the training set to determine the initial top-layer classifier. Re-parameterization of the first-order gradient descent approximation promotes the gradient back-propagation. Experiments on synthetic and benchmark data sets validate that our Aviator approach achieves the state-of-the-art performance, and visualization results demonstrate the task-adaptive features of our proposed Aviator method.

Few-Shot Learning for Image-Based Nonintrusive Appliance Signal Recognition

“…x_testsia[target_index:target_index + 1, 0].shape Out [32]: (1,1,200,200) In [34] In triplet loss Siamese model, the train set shape is (98, 28, 28, 1) and the test shape is (42,28,28,1). On the other hand, a part test code for the triplet loss is In [15]: btch_size � 9 epchs � 200 steps_per_epch � int(x_train.shape[0]/btch_size) We give both the contrastive and triplet loss plots in Section 4.4.…”

Prototypical Network Based on Manhattan Distance

“…There are more than 1000 samples in each category, which is very large. MiniImageNet selected 100 categories [46], including birds, animals, people, daily necessities, etc. Each category includes 600 84 * 84 RGB color pictures [47].…”

Fighting fire with fire: A spatial–frequency ensemble relation network with generative adversarial learning for adversarial image classification

“…Here, we mainly review the works of meta-learning-based relation learning and GANs. Meta-learning-based relation learning: Recently there are two types of meta-learning-based relation learning models 30,31 : (1) metrics-based methodologies 32,33 and (2) meta-optimizerbased methodologies. 34 The former is used to learn valid metrics and the corresponding matching function in a set of training examples.…”