Consecutive layer collaborative filter similarity for differentiable neural network pruning

Efficient tensor decomposition-based filter pruning

“…Comparison. CORING is compared with 44 SOTAs in the fields of structured pruning [1,3,5,6,7,11,12,15,16,18,19,23,26,29,30,31,32,34,35,36,37,38,39,40,41,43,45,46,47,48,49,56,58,63,64,66,68,73,76,78,79,80,83,85]. For a fair comparison, all available baseline models are identical.…”

“…However, these approaches neglect the correlation between inter-filters, resulting in high redundancy. Recent advancements [61,62,63,71,79,85] have demonstrated the potential benefits of leveraging the correlations or similarities between filters/feature maps filters from the network [1,29,38,63]. Unlike unstructured pruning, this method directly reduces the number of computations needed during the inference phase, leading to significant reductions in the network's memory footprint by directly decreasing the number of parameters.…”

“…To achieve this, filters are compared to each other based on a similarity metric, and those with the highest similarity are removed during the pruning process. Several works tried to apply common similarity metrics to measure filters correlation, such as Euclide [8,79], Manhattan [79], Chebyshev [61,79], Hamming [31] distances, cosine similarity [16,30,61,71,85], Pearson correlation [62,71], to mention a few. Most of these works overlook the multidimensional structure of filters and rely on flattened versions of filters to compute similarity, resulting in information loss.…”

IPBFS: Inference-aware Pruning with Bayesian Optimization based on Filter Similarity

“…The network learns from the similarity information during the optimization process and determines the optimal number of filters for pruning based on this information [34]. For this purpose, Bayesian optimization has been utilized to automatically determine the optimal count of filters based on the similarity of the feature map for pruning.…”