A survey of deep meta-learning

Huisman, Mike; Rijn, Jan N. van; Plaat, Aske

doi:10.1007/s10462-021-10004-4

Cited by 253 publications

(97 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In future work, we will first aim at improving the transferability of the remote sensing pre-trained models and work on covering the widely used image segmentation task. A more sophisticated transfer learning method, deep meta-learning [10], or customized techniques per dataset & task (based on [24,29]) integrated into AutoML systems could improve the usage of remote sensing data representations. Based on our experiments, we recommend the remote sensing practitioners to make use of the existing open-source AutoML tools.…”

Section: Discussionmentioning

confidence: 99%

Automated Machine Learning for Satellite Data: Integrating Remote Sensing Pre-trained Models into AutoML Systems

Salinas

Baratchi

Rijn

et al. 2021

Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track

Self Cite

View full text Add to dashboard Cite

Current AutoML systems have been benchmarked with traditional natural image datasets. Differences between satellite images and natural images (e.g., bit-wise resolution, the number, and type of spectral bands) and lack of labeled satellite images for training models, pose open questions about the applicability of current AutoML systems on satellite data. In this paper, we demonstrate how AutoML can be leveraged for classification tasks on satellite data. Specifically, we deploy the Auto-Keras system for image classification tasks and create two new variants, IMG-AK and RS-AK, for satellite image classification that respectively incorporate transfer learning using models pre-trained with (i) natural images (using ImageNet) and (ii) remote sensing datasets. For evaluation, we compared the performance of these variants against manually designed architectures on a benchmark set of 7 satellite datasets. Our results show that in 71% of the cases the AutoML systems outperformed the best previously proposed model, highlighting the usefulness of a customized satellite data search space in AutoML systems. Our RS-AK variant performed better than IMG-AK for small datasets with a limited amount of training data. Furthermore, it found the best automated model for the datasets composed of near-infrared, green, and red bands.

show abstract

Section: Discussionmentioning

confidence: 99%

Automated Machine Learning for Satellite Data: Integrating Remote Sensing Pre-trained Models into AutoML Systems

Salinas

Baratchi

Rijn

et al. 2021

Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track

Self Cite

View full text Add to dashboard Cite

show abstract

“…Few-shot learning was proposed to solve the problem of learning new classes in classifiers, where each class provides only a small number of training samples [30][31][32]. As a result of the development of deep learning techniques, the existing FSL methods can in the following: metric learning [44][45][46][47], which learns metrics/similarity of few-shot samples through deep networks; meta-learning [48][49][50], which learns a meta-model in multiple FSL tasks, and then the meta-model can be used to predict the weight of the model in a new FSL task; transfer learning [51,52], which uses pretrained weights as initialization; and data augmentation [53,54], which is a form of weak supervision [55] and aims to expand the few-shot sample dataset with additional data points. It should be noted that there is no absolute distinction between the four categories.…”

Section: Few-shot Learningmentioning

confidence: 99%

Optimizing Few-Shot Learning Based on Variational Autoencoders

Wei

Mahmood

2021

Entropy

View full text Add to dashboard Cite

Despite the importance of few-shot learning, the lack of labeled training data in the real world makes it extremely challenging for existing machine learning methods because this limited dataset does not well represent the data variance. In this research, we suggest employing a generative approach using variational autoencoders (VAEs), which can be used specifically to optimize few-shot learning tasks by generating new samples with more intra-class variations on the Labeled Faces in the Wild (LFW) dataset. The purpose of our research is to increase the size of the training dataset using various methods to improve the accuracy and robustness of the few-shot face recognition. Specifically, we employ the VAE generator to increase the size of the training dataset, including the basic and the novel sets while utilizing transfer learning as the backend. Based on extensive experimental research, we analyze various data augmentation methods to observe how each method affects the accuracy of face recognition. The face generation method based on VAEs with perceptual loss can effectively improve the recognition accuracy rate to 96.47% using both the base and the novel sets.

show abstract

“…In recent years, meta-learning methods have demonstrated promising results in various fields with different techniques [12]. Meta-learning techniques can be categorized into three groups [14][15][16]: metric-based methods [17][18][19], model-based methods [20][21][22][23], and optimization-based methods [8,24,25]. Optimization-based methods are often cast as a bilevel optimization problem and exhibit relatively better generalizability for wider task distributions.…”

Section: Related Workmentioning

confidence: 99%

“…We mainly focus on optimization-based meta-learning in this paper. For more comprehensive literature reviews and developments of meta-learning, we refer the readers to the recent surveys [12,16].…”

Section: Related Workmentioning

confidence: 99%

An Optimization-Based Meta-Learning Model for MRI Reconstruction with Diverse Dataset

Bian

Chen

et al. 2021

J. Imaging

View full text Add to dashboard Cite

This work aims at developing a generalizable Magnetic Resonance Imaging (MRI) reconstruction method in the meta-learning framework. Specifically, we develop a deep reconstruction network induced by a learnable optimization algorithm (LOA) to solve the nonconvex nonsmooth variational model of MRI image reconstruction. In this model, the nonconvex nonsmooth regularization term is parameterized as a structured deep network where the network parameters can be learned from data. We partition these network parameters into two parts: a task-invariant part for the common feature encoder component of the regularization, and a task-specific part to account for the variations in the heterogeneous training and testing data. We train the regularization parameters in a bilevel optimization framework which significantly improves the robustness of the training process and the generalization ability of the network. We conduct a series of numerical experiments using heterogeneous MRI data sets with various undersampling patterns, ratios, and acquisition settings. The experimental results show that our network yields greatly improved reconstruction quality over existing methods and can generalize well to new reconstruction problems whose undersampling patterns/trajectories are not present during training.

show abstract

A survey of deep meta-learning

Cited by 253 publications

References 33 publications

Automated Machine Learning for Satellite Data: Integrating Remote Sensing Pre-trained Models into AutoML Systems

Automated Machine Learning for Satellite Data: Integrating Remote Sensing Pre-trained Models into AutoML Systems

Optimizing Few-Shot Learning Based on Variational Autoencoders

An Optimization-Based Meta-Learning Model for MRI Reconstruction with Diverse Dataset

Contact Info

Product

Resources

About