Guided CNN for generalized zero-shot and open-set recognition using visual and semantic prototypes

Geng, Chuanxing; Tao, Lue; Chen, Songcan

doi:10.1016/j.patcog.2020.107263

Cited by 34 publications

(14 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second-level soft attention α is computed by using J matrix and K network similar to the first part (P I) of (2) i.e., α = softmax(tanh(J W B )W A ), where W B and W A are learned parameters of K network. The feature embedding F2 ∈ R r×m is constructed by summation of F1 and F 1 = α F1 as (3). Note that, since the information of the most relevant attribute to a region is propagated into the second-level and beyond through the embedding F1 , we do not use {T i } r i=1 neural networks in the second-level.…”

Section: Attribute Guided Attention Networkmentioning

confidence: 99%

“…Though, the underlying distribution of source and target domains is disjoint, the ZSL setting assumes that the trained visual classifier knows whether a test sample belongs to a source or target class. To alleviate such an unrealistic assumption, the ZSL setting is extended to a more realistic setting called Generalized Zero-Shot Learning (GZSL) [2,3,4], where the classifier has to classify test images from both source and target classes. The ultimate aim of this work is to improve GZSL for fine-grained recognition.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Integrated generalized zero-shot learning for fine-grained classification

Shermin

Teng

Sohel

et al. 2022

Pattern Recognition

View full text Add to dashboard Cite

Embedding learning (EL) and feature synthesizing (FS) are two of the popular categories of fine-grained GZSL methods. EL or FS using global features cannot discriminate fine details in the absence of local features. On the other hand, EL or FS methods exploiting local features either neglect direct attribute guidance or global information. Consequently, neither method performs well.In this paper, we propose to explore global and direct attribute-supervised local visual features for both EL and FS categories in an integrated manner for finegrained GZSL. The proposed integrated network has an EL sub-network and a FS sub-network. Consequently, the proposed integrated network can be tested in two ways. We propose a novel two-step dense attention mechanism to discover attribute-guided local visual features. We introduce new mutual learning between the sub-networks to exploit mutually beneficial information for optimization. Moreover, we propose to compute source-target class similarity based on mutual information and transfer-learn the target classes to reduce bias towards the source domain during testing. We demonstrate that our proposed method outperforms contemporary methods on benchmark datasets.

show abstract

Section: Attribute Guided Attention Networkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Integrated generalized zero-shot learning for fine-grained classification

Shermin

Teng

Sohel

et al. 2022

Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…LATEM [24] embedded the visual features with a piecewise linear function which was trained by a ranking based objective function. To alleviate the hubness problem [1] in the visual-to-semantic embedding methods, some works [2,31,32,33] proposed to learn a semantic-to-visual embedding, where they mapped semantic features into visual features by a regression function. DEM [1] used a two-layer neural network to learn a discriminative visual features space from the semantic feature space.…”

Section: Zero-shot Learningmentioning

confidence: 99%

Semantic-diversity transfer network for generalized zero-shot learning via inner disagreement based OOD detector

Liu¹,

Dong²,

Hu³

2022

Preprint

View full text Add to dashboard Cite

Zero-shot learning (ZSL) aims to recognize objects from unseen classes, where the kernel problem is to transfer knowledge from seen classes to unseen classes by establishing appropriate mappings between visual and semantic features.The knowledge transfer in many existing works is limited mainly due to the facts that (i) the widely used visual features are global ones but not totally consistent with semantic attributes; (ii) only one mapping is learned in existing works, which is not able to effectively model diverse visual-semantic relations;(iii) the bias problem in the generalized ZSL (GZSL) could not be effectively handled. In this paper, we propose two techniques to alleviate these limitations.Firstly, we propose a Semantic-diversity transfer Network (SetNet) addressing the first two limitations, where 1) a multiple-attention architecture and a diversity regularizer are proposed to learn multiple local visual features that are more consistent with semantic attributes and 2) a projector ensemble that geometrically takes diverse local features as inputs is proposed to model visual-semantic relations from diverse local perspectives. Secondly, we propose an inner dis-

show abstract

“…Outliers are usually rejected by thresholding some kind of score. Distance based classifiers classify a sample as an outlier if it is far from any of the learned prototypes [15].…”

Section: Open-set Recognitionmentioning

confidence: 99%

Dense outlier detection and open-set recognition based on training with noisy negative images

Bevandić¹,

Krešo²,

Oršić³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep convolutional models often produce inadequate predictions for inputs foreign to the training distribution. Consequently, the problem of detecting outlier images has recently been receiving a lot of attention. Unlike most previous work, we address this problem in the dense prediction context in order to be able to locate outlier objects in front of in-distribution background. Our approach is based on two reasonable assumptions. First, we assume that the inlier dataset is related to some narrow application field (e.g. road driving). Second, we assume that there exists a general-purpose dataset which is much more diverse than the inlier dataset (e.g. ImageNet-1k). We consider pixels from the general-purpose dataset as noisy negative training samples since most (but not all) of them are outliers. We encourage the model to recognize borders between known and unknown by pasting jittered negative patches over inlier training images. Our experiments target two dense open-set recognition benchmarks (WildDash 1 and Fishyscapes) and one dense open-set recognition dataset (StreetHazard). Extensive performance evaluation indicates competitive potential of the proposed approach.

show abstract

Guided CNN for generalized zero-shot and open-set recognition using visual and semantic prototypes

Cited by 34 publications

References 10 publications

Integrated generalized zero-shot learning for fine-grained classification

Integrated generalized zero-shot learning for fine-grained classification

Semantic-diversity transfer network for generalized zero-shot learning via inner disagreement based OOD detector

Dense outlier detection and open-set recognition based on training with noisy negative images

Contact Info

Product

Resources

About