Few-Shot Unsupervised Domain Adaptation via Meta Learning

Abstract

Few-shot unsupervised domain adaptation (FS-UDA) utilizes few-shot labeled source domain data to realize effective classification in unlabeled target domain. However, current FS-UDA methods are still suffer from two issues: 1) the data from different domains can not be effectively aligned by few-shot labeled data due to the large domain gaps, 2) it is unstable and time-consuming to generalize to new FS-UDA tasks. To address this issue, we put forward a novel Efficient Meta Prompt Learning Framework for FS-UDA. Within this framework, we use pre-trained CLIP model as the feature learning base model. First, we design domain-shared prompt learning vectors composed of virtual tokens, which mainly learns the meta knowledge from a large number of meta tasks to mitigate domain gaps. Secondly, we also design a task-shared prompt learning network to adaptively learn specific prompt vectors for each task, which aims to realize fast adaptation and task generalization. Thirdly, we learn a task-specific crossdomain alignment projection and a task-specific classifier with closed-form solutions for each meta task, which can efficiently adapt the model to new tasks in one step. The whole learning process is formulated as a bilevel optimization problem, and a good initialization of model parameters is learned through metalearning. Extensive experimental study demonstrates the promising performance of our framework on benchmark datasets. Our method has the large improvement of at least 15.4% on 5-way 1-shot and 8.7% on 5-way 5-shot, compared with the state-ofthe-art methods. Also, the performance of our method on all the test tasks is more stable than the other methods.

Advances and Challenges in Meta-Learning: A Technical Review

“…Additionally, they assume the input-output relationship (i.e., p(y|x)) to be the same across domains. To solve these problems, some methods [155,156,157,153] Effective domain generalization via meta-learning: Domain generalization enables models to perform well on new and unseen domains without requiring access to their data, as illustrated in Figure 8. This is particularly useful in scenarios where access to data is restricted due to real-time deployment requirements or privacy policies.…”

High-Level Semantic Feature Matters Few-Shot Unsupervised Domain Adaptation

“…Optimization-based methods (Bertinetto et al 2019) (Finn, Abbeel, and Levine 2017) (Ravi and Larochelle 2017) usually train a meta learner over auxiliary dataset to learn a general initialization model, which can fine-tune and adapt to new tasks very soon. The main purpose of metricbased methods (Li et al 2019) (Snell, Swersky, and Zemel 2017) (Vinyals et al 2016) (Ye et al 2020) is that learn a generalizable feature embedding for metric learning, which can immediately adapt to new tasks without any fine-tune and retraining. Typically, ProtoNet (Snell, Swersky, and Zemel 2017) learns the class prototypes in the support set and classifies the query images based on the maximum similarity to these prototypes.…”

“…Currently, a setting namely few-shot unsupervised domain adaptation (FS-UDA) (Huang et al 2021) (Yang et al 2022), which utilizes few labeled data in source domain to train a model to classify unlabeled data in target domain, owns its potential feasibility. Typically, a FS-UDA model could learn general knowledge from base classes during training to guide classification in novel classes during testing.…”

Zero-Shot Transfer Learning Framework for Plant Leaf Disease Classification

“…As a result, transfer learning is increasingly recognized as a vital approach for tackling practical challenges. Within the realm of transfer learning, two specialized techniques are zero-shot learning [29]- [32] and few-shot learning [33], [34]. These approaches address scenarios where there is limited or no labeled data available in the target domain.…”