HRank: Filter Pruning Using High-Rank Feature Map

Lin, Mingbao; Ji, Rongrong; Wang, Yan; Zhang, Yichen; Zhang, Baochang; Tian, Yonghong; Shao, Ling

doi:10.1109/cvpr42600.2020.00160

Cited by 652 publications

(532 citation statements)

References 20 publications

Supporting

Mentioning

523

Contrasting

Unclassified

Order By: Relevance

“…Our method is compared to the classical first-k and max response [20], the state-of-the-art channel pruning [21], Thin Net [35] and HRank [30] that are similar to our method to some extent.…”

Section: Methodsmentioning

confidence: 99%

“…Similar to the above channel pruning method [21], some other filter-level pruning methods [12,20,30,35] also have been explored. The core of the filter pruning is to measure the importance of each filter.…”

Section: Introductionmentioning

confidence: 99%

“…It is worth noting that both channel pruning method and ThiNet method are driven by data to demonstrate the effectiveness of filter selection strategy, and first k and max response are non-data-driven methods. Besides, HRank [30], as a data-driven method, is proposed as follows:…”

Section: Introductionmentioning

confidence: 99%

“…There has been rising interest in building small and efficient neural networks in the recent literature [17][18][19][20][21][22][23][24][25][26][27][28]. Many different approaches can be generally categorized as two groups: (1) training small networks directly [22][23][24][25][26][27]29]; (2) compressing pre-trained networks [17][18][19][20][21]28,30].…”

mentioning

confidence: 99%

See 3 more Smart Citations

Filter pruning-based two-step feature map reconstruction

Liang

Liu

et al. 2021

SIViP

View full text Add to dashboard Cite

In deep neural network compression, channel/filter pruning is widely used for compressing the pre-trained network by judging the redundant channels/filters. In this paper, we propose a two-step filter pruning method to judge the redundant channels/filters layer by layer. The first step is to design a filter selection scheme based on $$\ell _{2,1}$$ ℓ 2 , 1 -norm by reconstructing the feature map of current layer. More specifically, the filter selection scheme aims to solve a joint $$\ell _{2,1}$$ ℓ 2 , 1 -norm minimization problem, i.e., both the regularization term and feature map reconstruction error term are constrained by $$\ell _{2,1}$$ ℓ 2 , 1 -norm. The $$\ell _{2,1}$$ ℓ 2 , 1 -norm regularization plays a role in the channel/filter selection, while the $$\ell _{2,1}$$ ℓ 2 , 1 -norm feature map reconstruction error term plays a role in the robust reconstruction. In this way, the proposed filter selection scheme can learn a column-sparse coefficient representation matrix that can indicate the redundancy of filters. Since pruning the redundant filters in current layer might dramatically influence the output feature map of the following layer, the second step needs to update the filters of the following layer to assure output of feature map approximates to that of baseline. Experimental results demonstrate the effectiveness of this proposed method. For example, our pruned VGG-16 on ImageNet achieves $$4\times $$ 4 × speedup with 0.95% top-5 accuracy drop. Our pruned ResNet-50 on ImageNet achieves $$2\times $$ 2 × speedup with 1.56% top-5 accuracy drop. Our pruned MobileNet on ImageNet achieves $$2\times $$ 2 × speedup with 1.20% top-5 accuracy drop.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Filter pruning-based two-step feature map reconstruction

Liang

Liu

et al. 2021

SIViP

View full text Add to dashboard Cite

show abstract

“…数据驱动的剪枝算法有: Hu 等 [31] 提出了 APoZ 算法, 用于计算卷积核输出特征图 0 值的数量, 此算法认为输出特征图的 0 值越多其重要性越低, 然后将重要度低的特征图所对应的卷积核进行剪枝. Lin 等 [32] 提出动态剪枝算法, 在实现过程中可以根据实际剪枝情况, 对网络做细致的调节. Lin 等 [33] 提出 HRank 算法, 该算法采用特征图的秩表达相应卷积核的重要度. Wang 等 [34] 对正向传播中的特征图进行子空间聚类, 同时根据聚类结果和已经设置的条件去除相应的卷积核.…”

Section: 相关工作unclassified

Convolution network pruning based on the evaluation of the importance of characteristic attributions

Zhang¹,

Yang²,

Sang³

et al. 2020

Sci. Sin.-Inf.

View full text Add to dashboard Cite

show abstract

Retracted : Optimizing the convolutional network layers using the Viola–Jones framework and ternary weight networks

Al‐Gabalawy

2021

Int J Intell Syst

View full text Add to dashboard Cite

In this paper, we have presented a method of combining the weight optimizations from ternary weight networks with a set of custom features inspired by the Viola–Jones Object Detection Framework to improve a network's performance. By replacing the first convolutional layer in a network with a function using a custom set of filters, we can reduce the number of operations required to compute the layer's outputs. These filters are composed of four rectangles, labeled as either positive or negative, within the filter space. When combined with an integral image, these filters allow for more efficient computations of feature values than a standard convolutional layer. When using our custom filters of shape 5 × 5, the layer only requires 64% of the operations that a standard convolutional layer need. This reduction only increases as the filter sizes get larger. Empirically, we have found that these layers' real‐world execution times behave as theoretically calculated and that their optimizations do not lead to significant drop inaccuracy. During testing, we evaluated our proposed optimization on two different data sets with two different network architectures.

show abstract

HRank: Filter Pruning Using High-Rank Feature Map

Cited by 652 publications

References 20 publications

Filter pruning-based two-step feature map reconstruction

Filter pruning-based two-step feature map reconstruction

Convolution network pruning based on the evaluation of the importance of characteristic attributions

Retracted : Optimizing the convolutional network layers using the Viola–Jones framework and ternary weight networks

Contact Info

Product

Resources

About