Distribution Consistency Based Covariance Metric Networks for Few-Shot Learning

Abstract

Few-shot learning aims to recognize new concepts from very few examples. However, most of the existing few-shot learning methods mainly concentrate on the first-order statistic of concept representation or a fixed metric on the relation between a sample and a concept. In this work, we propose a novel end-to-end deep architecture, named Covariance Metric Networks (CovaMNet). The CovaMNet is designed to exploit both the covariance representation and covariance metric based on the distribution consistency for the few-shot classification tasks. Specifically, we construct an embedded local covariance representation to extract the second-order statistic information of each concept and describe the underlying distribution of this concept. Upon the covariance representation, we further define a new deep covariance metric to measure the consistency of distributions between query samples and new concepts. Furthermore, we employ the episodic training mechanism to train the entire network in an end-to-end manner from scratch. Extensive experiments in two tasks, generic few-shot image classification and fine-grained fewshot image classification, demonstrate the superiority of the proposed CovaMNet. The source code can be available from https://github.com/WenbinLee/CovaMNet.git.

Compare More Nuanced: Pairwise Alignment Bilinear Network for Few-Shot Fine-Grained Learning

“…Finally, we analyze the experimental results of the proposed models and compare them with other few-shot learning approaches. For a fair comparison, we conduct two groups of experiments on these datasets, for the first group, we follow the setting, which Wei et al [1], [23] used, while for the second group, we follow the newest settings in the recent few-shot methods [19], [20].…”

“…• DN4 [20], the newest generic few-shot method published in CVPR 2019. By using a deep nearest neighbor neural netwok, DN4 can aggregate the discriminative [19], the newest FSFG model published in AAAI 2019. It replaces the bilinear pooling with covariance bilinear pooling and achieves state-of-the-art performance on FSFG classification.…”

“…Then with the operation of bilinear feature grouping, the PCM model can generate low-rank subtle descriptors of the original image. Most recently, Li et al [19] propose to uses the covariance pooling [10] to distillate the image representation of each category. These matrix-outer-product based bilinear pooling operations could extract the secondorder image features and contains more information than traditional first-order features [24], and thus achieve better performance on fine-grained data than generic ones.…”

“…To address the above issues, we propose a novel endto-end FSFG model that captures the fine-grained relations among different classes. This subtle comparative ability of our models is inherently more intelligent than merely modeling the data distribution [17], [19], [23]. The main contributions are summarized as follows:…”

“…• Pairwise Bilinear Pooling. Existing second-order based FSFG methods [19], [23] enhance the encoded individual features by directly applying the self-bilinear pooling operation. However, such an operation fails to capture more nuanced relations between similar objects.…”