Semantically Consistent Regularization for Zero-Shot Recognition

Morgado, Pedro; Vasconcelos, Nuno

doi:10.1109/cvpr.2017.220

Cited by 139 publications

(94 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zero-shot action recognition With the explosion of videos and the difficulty of scalability of manual annotation, there have been approaches to recognize the unknown action in the way of zero-shot. ZSAR associates known categories and unknown categories based on semantic space, including manual attributes [23,24,25,26,27], text descriptions [28] and word vectors [29,30]. However, attributes are not easy to define and the manually-specified attributes are highly subjective.…”

Section: Related Workmentioning

confidence: 99%

Action Recognition in Untrimmed Videos with Composite Self-attention Two-Stream Framework

Cao¹,

Xu²,

Chen

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

With the rapid development of deep learning algorithms, action recognition in video has achieved many important research results. One issue in action recognition, Zero-Shot Action Recognition (ZSAR), has recently attracted considerable attention, which classify new categories without any positive examples. Another difficulty in action recognition is that untrimmed data may seriously affect model performance. We propose a composite two-stream framework with a pre-trained model. Our proposed framework includes a classifier branch and a composite feature branch. The graph network model is adopted in each of the two branches, which effectively improves the feature extraction and reasoning ability of the framework. In the composite feature branch, a 3-channel self-attention models are constructed to weight each frame in the video and give more attention to the key frames. Each selfattention models channel outputs a set of attention weights to focus on a particular aspect of the video, and a set of attention weights corresponds to a one-dimensional vector. The 3-channel self-attention models can evaluate key frames from multiple aspects, and the output sets of attention weight vectors form an attention matrix, which effectively enhances the attention of key frames with strong correlation of action. This model can implement action recognition under zero-shot conditions, and has good recognition performance for untrimmed video data. Experimental results on relevant data sets confirm the validity of our model.

show abstract

Section: Related Workmentioning

confidence: 99%

Action Recognition in Untrimmed Videos with Composite Self-attention Two-Stream Framework

Cao¹,

Xu²,

Chen

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Multitask learning is sometimes also used to solve auxiliary tasks that strengthen performance of a task of interest, e.g. by accounting for context [10], or representing objects in terms of classes and attributes [15,29,30,25].…”

Section: Related Workmentioning

confidence: 99%

Efficient Multi-Domain Learning by Covariance Normalization

Vasconcelos

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

View full text Add to dashboard Cite

The problem of multi-domain learning of deep networks is considered. An adaptive layer is induced per target domain and a novel procedure, denoted covariance normalization (CovNorm), proposed to reduce its parameters. CovNorm is a data driven method of fairly simple implementation, requiring two principal component analyzes (PCA) and fine-tuning of a mini-adaptation layer. Nevertheless, it is shown, both theoretically and experimentally, to have several advantages over previous approaches, such as batch normalization or geometric matrix approximations. Furthermore, CovNorm can be deployed both when target datasets are available sequentially or simultaneously. Experiments show that, in both cases, it has performance comparable to a fully fine-tuned network, using as few as 0.13% of the corresponding parameters per target domain.

show abstract

“…It builds up the relations between seen and unseen classes, thus making it possible for zero-shot recognition. Recently, the most popular semantic information includes attributes [8,9,12,17,18] and wordvectors [11,19,20]. Attributes are general descriptions of objects which can be shared among different classes.…”

Section: Semantic Informationmentioning

confidence: 99%

Learning Class Prototypes via Structure Alignment for Zero-Shot Recognition

Jiang

Wang

Shan

et al. 2018

Lecture Notes in Computer Science

106

View full text Add to dashboard Cite

Zero-shot learning (ZSL) aims to recognize objects of novel classes without any training samples of specific classes, which is achieved by exploiting the semantic information and auxiliary datasets. Recently most ZSL approaches focus on learning visual-semantic embeddings to transfer knowledge from the auxiliary datasets to the novel classes. However, few works study whether the semantic information is discriminative or not for the recognition task. To tackle such problem, we propose a coupled dictionary learning approach to align the visual-semantic structures using the class prototypes, where the discriminative information lying in the visual space is utilized to improve the less discriminative semantic space. Then, zero-shot recognition can be performed in different spaces by the simple nearest neighbor approach using the learned class prototypes. Extensive experiments on four benchmark datasets show the effectiveness of the proposed approach.

show abstract

Semantically Consistent Regularization for Zero-Shot Recognition

Cited by 139 publications

References 49 publications

Action Recognition in Untrimmed Videos with Composite Self-attention Two-Stream Framework

Action Recognition in Untrimmed Videos with Composite Self-attention Two-Stream Framework

Efficient Multi-Domain Learning by Covariance Normalization

Learning Class Prototypes via Structure Alignment for Zero-Shot Recognition

Contact Info

Product

Resources

About