Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition

Abstract

Many state-of-the-art few-shot learners focus on developing effective training procedures for feature representations, before using simple (e.g., nearest centroid) classifiers. We take an approach that is agnostic to the features used, and focus exclusively on meta-learning the final classifier layer. Specifically, we introduce MetaQDA, a Bayesian meta-learning generalisation of the classic quadratic discriminant analysis. This approach has several benefits of interest to practitioners: meta-learning is fast and memory efficient, without the need to fine-tune features. It is agnostic to the off-the-shelf features chosen, and thus will continue to benefit from future advances in feature representations. Empirically, it leads to excellent performance in cross-domain few-shot learning, class-incremental few-shot learning, and crucially for real-world applications, the Bayesian formulation leads to state-of-the-art uncertainty calibration in predictions.

Adaptive Transformers for Robust Few-shot Cross-domain Face Anti-spoofing

“…Few-shot learning methods [20,63,2,55,56,60,15,51] aim to adapt models to novel classes from a few samples from each class (assuming the classes used for training are disjoint with the novel classes seen at test time). Cross-domain few-shot learning [78,19,58] further addresses the problem when the novel classes are sampled from a different domain with different data distribution. In contrast, few-shot supervised domain adaptation aims to adapt models to new domains with the assistance of a few examples [44,57,65,45].…”

Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference

“…Much activity in the field falls under the umbrella of meta-learning [35], which aims to construct a data-efficient learner from the source (aka meta-train) dataset by simulating few-shot learning problems, and then deploy the customized learner on the target (aka meta-test) set. The resulting learner may take the form of an initialization [29], learned metric [59], Bayesian prior [72], or optimizer [54]. Simple-but-effective baselines In competition with the plethora of sophisticated few-shot learners [35,69] such as those mentioned above, a number of recent studies have advocated strong baselines that perform comparably well while being simpler.…”

“…How does pre-training and architecture affect fewshot learning? Learning from a few shots can be achieved by a) meta-learning [66,72] and b) transfer learning from self-supervised foundation models pre-trained on large-scale external data [18,53]. While the majority of FSL community focuses on the former, we show that the latter can be more effective because it enables the use of stronger architectures such as vision transformer (ViT) [25] -and can be combined with simple meta-learners such as ProtoNet.…”

“…To emphasize the importance of pre-training, ProtoNet + randomly initialized ViT (blue square) is also compared. pernetworks [9], neural optimizers [54], transductive label propagation [55], neural loss learning [4], Bayesian neural priors [72] and more [69]. But how much practical progress have we made based on all these technical advances?…”

“…A few studies [19,20,23,51,63,68] have investigated whether simpler baselines can offer comparable performance to sophisticated state of the art few-shot learners. While there is no conclusive answer, due to on-going developments in both sophisticated learners [72] and simple baselines, there is a trend that simple approaches often perform surprisingly well compared to sophisticated counterparts. Their simplicity and efficacy leads these simple methods to be taken up in many practical applications of few-shot learning from medical data analysis [11] to electronic engineering [40].…”