2019
DOI: 10.1609/aaai.v33i01.33014683
|View full text |Cite
|
Sign up to set email alerts
|

Compressing Recurrent Neural Networks with Tensor Ring for Action Recognition

Abstract: Recurrent Neural Networks (RNNs) and their variants, such as Long-Short Term Memory (LSTM) networks, and Gated Recurrent Unit (GRU) networks, have achieved promising performance in sequential data modeling. The hidden layers in RNNs can be regarded as the memory units, which are helpful in storing information in sequential contexts. However, when dealing with high dimensional input data, such as video and text, the input-to-hidden linear transformation in RNNs brings high memory usage and huge computational co… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
2
1
1
1

Citation Types

0
81
0

Year Published

2020
2020
2023
2023

Publication Types

Select...
4
3

Relationship

1
6

Authors

Journals

citations
Cited by 96 publications
(81 citation statements)
references
References 28 publications
0
81
0
Order By: Relevance
“…Thus, for fairer comparison and validation of search results, we implement this experiment in Pytorch and remove the tricks in the Keras package. Additionally, through Keras implement, our searched TR-LSTM achieve 64.5% accuracy with a compression ratio 48, which is better than 63.8% with a compression ratio 25 [17].…”
Section: Experiments On Hmdb51 and Ucf11mentioning
confidence: 91%
See 3 more Smart Citations
“…Thus, for fairer comparison and validation of search results, we implement this experiment in Pytorch and remove the tricks in the Keras package. Additionally, through Keras implement, our searched TR-LSTM achieve 64.5% accuracy with a compression ratio 48, which is better than 63.8% with a compression ratio 25 [17].…”
Section: Experiments On Hmdb51 and Ucf11mentioning
confidence: 91%
“…Wenqi et al [26] compress both the fully connected layers and the convolutional layers of CNN with the equal rank elements for whole network. Yu et al [17] replace the over-parametric input-to-hidden layer of LSTM with TRF, when dealing with high-dimensional input data. Rank of these models are determined via multiple manual attempts by manipulation, which requires much time.…”
Section: Rank Fixedmentioning
confidence: 99%
See 2 more Smart Citations
“…The tensor train format was employed in Novikov et al (2015) to reduce the parameters in fully connected layers. Several tensor decomposition methods were also applied to compress RNNs (Tjandra, Sakti, and Nakamura 2018;Ye et al 2018;Pan et al 2019). In spite of the empirical success of low-rank matrix and tensor approaches, theoretical studies for learning efficiency are still limited.…”
Section: Introductionmentioning
confidence: 99%