Multi-Level Semantic Feature Augmentation for One-Shot Learning

Chen, Zitian; Fu, Yanwei; Zhang, Yinda; Jiang, Yu‐Gang; Xue, Xiangyang; Sigal, Leonid

doi:10.1109/tip.2019.2910052

Cited by 243 publications

(109 citation statements)

References 60 publications

(103 reference statements)

Supporting

Mentioning

109

Contrasting

Order By: Relevance

“…[8], CP-ANN [6], PMN, and PMN w/ H [30]. (3) The data augmentation methods are also compared -flipping: the input image is flipped from left to right; Gaussian noise: cross-validated Gaussian noise N (0, 10) is added to each pixel of the input image; Gaussian noise (feature level): cross-validated Gaussian noise N (0, 0.3) is added to each dimension of the ResNet feature for each image; Mixup: using mixup [35] to combine probe and gallery images. For fair comparisons, all these augmentation methods use the prototype classifier as the one-shot classifier.…”

Section: Results On Imagenet 1k Challengementioning

confidence: 99%

“…However, the generated images in this way are particularly subject to visual similarity with the original images. In addition to adding noise or jittering, previous work seeks to augment training images by using semi-supervised techniques [31,18,16] and utilizing relation between visual and semantic representations [3] , or directly synthesizing new instances in the feature domain [9,30,21,6] to transfer knowledge of data distribution from base classes to novel classes. By contrast, we also use samples from base classes to help synthesize deformed images but directly aim at maximizing the one-shot recognition accuracy.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Image Deformation Meta-Networks for One-Shot Learning

Chen

Wang

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

216

View full text Add to dashboard Cite

Figure 1. Illustration of a variety of image deformations: ghosted (a, b), stitched (c), montaged (d), and partially occluded (e) images. AbstractHumans can robustly learn novel visual concepts even when images undergo various deformations and lose certain information. Mimicking the same behavior and synthesizing deformed instances of new concepts may help visual recognition systems perform better one-shot learning, i.e., learning concepts from one or few examples. Our key insight is that, while the deformed images may not be visually realistic, they still maintain critical semantic information and contribute significantly to formulating classifier decision boundaries. Inspired by the recent progress of meta-learning, we combine a meta-learner with an image deformation sub-network that produces additional training examples, and optimize both models in an end-to-end manner. The deformation sub-network learns to deform images by fusing a pair of images -a probe image that keeps the visual content and a gallery image that diversifies the deformations. We demonstrate results on the widely used one-shot learning benchmarks (miniImageNet and Im-ageNet 1K Challenge datasets), which significantly outperform state-of-the-art approaches. Code is available at https://github.com/tankche1/IDeMe-Net.

show abstract

Section: Results On Imagenet 1k Challengementioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Image Deformation Meta-Networks for One-Shot Learning

Chen

Wang

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

216

View full text Add to dashboard Cite

show abstract

“…To address this critical problem, a classifier should be implemented to distinguish the new categories. To the best of our knowledge, current researches commonly choose one of the following classifiers to gain their best performance, such as SVM [53], cosine-similarity [47] and nearest neighbor.…”

Section: ) Convnet-based Feature Extractormentioning

confidence: 99%

“…However, the miniImageNet consists of 60,000 color images with 100 classes of which 64 classes for training. The training data is enough to learn a good feature extractor for a common few-shot classification task, and nearly 80% accuracy has been already achieved recently [53]. In this paper, we focus on the fine-grained few-shot classification tasks.…”

Section: A Experimental Designmentioning

confidence: 99%

Few-Shot Learning for Domain-Specific Fine-Grained Image Classification

Sun

Dong

et al. 2021

IEEE Trans. Ind. Electron.

View full text Add to dashboard Cite

Learning to recognize novel visual categories from a few examples is a challenging task for machines in realworld applications. In contrast, humans have the ability to discriminate even similar objects with little supervision. This paper attempts to address the few-shot fine-grained recognition problem. We propose a feature fusion model to explore the largest discriminative features by focusing on key regions. The model utilizes focus-area location to discover the perceptually similar regions among objects. High-order integration is employed to capture the interaction information among intra-parts. We also design a Center Neighbor Loss to form robust embedding space distribution for generating discriminative features. Furthermore, we build a typical fine-grained and few-shot learning dataset miniPPlankton from the real-world application in the area of marine ecological environment. Extensive experiments are carried out to validate the performance of our model. First the model is evaluated with two challenging experiments based on the miniDogsNet and Caltech-UCSD public datasets. The results demonstrate that our model achieves competitive performance compared with state-of-the-art models. Then, we implement our model for the real-world phytoplankton recognition task. The experimental results show the superiority of the proposed model compared with others on the miniPPlankton dataset.

show abstract

“…Meta Network proposed by Munkhdalai et al [40] allows fast generalization through shift of inductive bias using meta learner. Besides, lots of researchers try to generate auxiliary samples using a few samples in a class to alleviate overfitting [41][42][43] where Generative Adversarial Network (GAN) is commonly used. Generating process called hallucination is often guided by attribute, semantics and regularization to reduce deviation between results and original samples.…”

Section: Few-shot Learningmentioning

confidence: 99%

ConvProtoNet: Deep Prototype Induction towards Better Class Representation for Few-Shot Malware Classification

Tang

Wang

2020

Applied Sciences

View full text Add to dashboard Cite

Traditional malware classification relies on known malware types and significantly large datasets labeled manually which limits its ability to recognize new malware classes. For unknown malware types or new variants of existing malware containing only a few samples each class, common classification methods often fail to work well due to severe overfitting. In this paper, we propose a new neural network structure called ConvProtoNet which employs few-shot learning to address the problem of scarce malware samples while prevent from overfitting. We design a convolutional induction module to replace the insufficient prototype reduction in most few-shot models and generates more appropriate class-level malware prototypes for classification. We also adopt meta-learning scheme to make classifier robust enough to adapt unseen malware classes without fine-tuning. Even in extreme conditions where only 5 samples in each class are provided, ConvProtoNet still achieves more than 70% accuracy on average and outperforms other traditional malware classification methods or existed few-shot models in experiments conducted on several datasets. Extra experiments across datasets illustrate that ConvProtoNet learns general knowledge of malware which is dataset-invariant and careful model analysis proves effectiveness of ConvProtoNet in few-shot malware classification.

show abstract

Multi-Level Semantic Feature Augmentation for One-Shot Learning

Cited by 243 publications

References 60 publications

Image Deformation Meta-Networks for One-Shot Learning

Image Deformation Meta-Networks for One-Shot Learning

Few-Shot Learning for Domain-Specific Fine-Grained Image Classification

ConvProtoNet: Deep Prototype Induction towards Better Class Representation for Few-Shot Malware Classification

Contact Info

Product

Resources

About