Exploring Inter-feature and Inter-class Relationships with Deep Neural Networks for Video Classification

Wu, Zuxuan; Jiang, Yu‐Gang; Wang, Jun; Pu, Jian; Xue, Xiangyang

doi:10.1145/2647868.2654931

Cited by 134 publications

(79 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The detailed analyses of these results are presented as follows. Hollywood-2 HMDB51 Jain et al [18] 62.5% 52.1% Oneata et al [14] 63.3% 54.8% Wang et al [12] 64.3% 57.2% Wu et al [19] 64.5% -Simonyan et al [20] -59.4% STFV+CombFV 66.72% 61.50% STFV+CombFV+ScaleFV 66.96% 61.07% may be due to the reason that STFV owns a larger dimension than ScaleFV, i.e., the dimension of STFV is (2 +1+7 ) , while the dimension of ScaleFV is (2 + 1 + 5 ) . Such a comparison shows that it may not be sufficient to consider the scale information solely for human action recognition, which inspires us to consider a more powerful encoding method to take both scale and spatial-temporal position information into account.…”

Section: Resultsmentioning

confidence: 99%

Encoding scale into fisher vector for human action recognition

Zhang

Wang

2015

2015 Visual Communications and Image Processing (VCIP)

View full text Add to dashboard Cite

In this paper, a new kind of Fisher Vector (FV) model, named Scale FV (ScaleFV), is proposed to ameliorate visual feature encoding for human action recognition. Although several researches have been proposed for feature encoding, the temporal scale information is almost ignored. Similar to the spatial scale information which has shown to be important in extracting and encoding visual features, the temporal scale information also plays an important role in video content analysis based on our investigation. To demonstrate this, a definition of temporal scale in videos is given, and it is presented that both of the spatial and temporal scale information can be encoded into the FV model by slightly modifying the underlying Gaussian Mixture Models (GMM). Furthermore, an enhanced FV model termed as Combined FV (CombFV) is designed to capture both position and scale information for human action recognition.Comparative experiments are carried out to demonstrate the superior performance of the proposed methods.

show abstract

Section: Resultsmentioning

confidence: 99%

Encoding scale into fisher vector for human action recognition

Zhang

Wang

2015

2015 Visual Communications and Image Processing (VCIP)

View full text Add to dashboard Cite

show abstract

“…Recently, deep learning technologies have been utilized in Web video classification and achieved significant performance improvement [25][26][27]. In [26], video classification system was presented using regularizations in deep neural networks.…”

Section: Web Video Classificationmentioning

confidence: 99%

“…In [26], video classification system was presented using regularizations in deep neural networks. Both features and class relationships were explored to obtain better video classification performance.…”

Section: Web Video Classificationmentioning

confidence: 99%

Web video categorization using category-predictive classifiers and category-specific concept classifiers

Afzal

Chen

et al. 2016

Neurocomputing

Self Cite

View full text Add to dashboard Cite

“…The equation can decrease the magnitude of the weights and is commonly used to prevent overfitting [47].…”

Section: The Frameworkmentioning

confidence: 99%

Learning Compact Hash Codes for Multimodal Representations Using Orthogonal Deep Structure

Wang

Cui

et al. 2015

IEEE Trans. Multimedia

View full text Add to dashboard Cite

As large-scale multimodal data are ubiquitous in many real-world applications, learning multimodal representations for efficient retrieval is a fundamental problem. Most existing methods adopt shallow structures to perform multimodal representation learning. Due to a limitation of learning ability of shallow structures, they fail to capture the correlation of multiple modalities. Recently, multimodal deep learning was proposed and had proven its superiority in representing multimodal data due to its high nonlinearity. However, in order to learn compact and accurate representations, how to reduce the redundant information lying in the multimodal representations and incorporate different complexities of different modalities in the deep models is still an open problem. In order to address the aforementioned problem, in this paper we propose a hashing-based orthogonal deep model to learn accurate and compact multimodal representations. The method can better capture the intra-modality and inter-modality correlations to learn accurate representations. Meanwhile, in order to make the representations compact, the hashing-based model can generate compact hash codes and the proposed orthogonal structure can reduce the redundant information lying in the codes by imposing orthogonal regularizer on the weighting matrices. We also theoretically prove that, in this case, the learned codes are guaranteed to be approximately orthogonal. Moreover, considering the different characteristics of different modalities, effective representations can be attained with different number of layers for different modalities. Comprehensive experiments on three real-world datasets demonstrate a substantial gain of our method on retrieval tasks compared with existing algorithms.

show abstract

Exploring Inter-feature and Inter-class Relationships with Deep Neural Networks for Video Classification

Cited by 134 publications

References 44 publications

Encoding scale into fisher vector for human action recognition

Encoding scale into fisher vector for human action recognition

Web video categorization using category-predictive classifiers and category-specific concept classifiers

Learning Compact Hash Codes for Multimodal Representations Using Orthogonal Deep Structure

Contact Info

Product

Resources

About