On the Role of Structured Pruning for Neural Network Compression

Abstract

This works explores the benefits of structured parameter pruning in the framework of the MPEG standardization efforts for neural network compression. First less relevant parameters are pruned from the network, then remaining parameters are quantized and finally quantized parameters are entropy coded. We consider an unstructured pruning strategy that maximizes the number of pruned parameters at the price of randomly sparse tensors and a structured strategy that prunes fewer parameters yet yields regularly sparse tensors. We show that structured pruning enables better end-to-end compression despite lower pruning ratio because it boosts the efficiency of the arithmetic coder. As a bonus, once decompressed, the network memory footprint is lower as well as its inference time.

Efficient Inference Of Image-Based Neural Network Models In Reconfigurable Systems With Pruning And Quantization

“…Pruning In recent years, the focus of NNs pruning, shifted from unstructured (parameters are removed independently) to structured (entire neurons are zeroed-out). A recent work [18] even showed that structured procedures greatly benefit endto-end compression and the deployment on embedded devices. Following this trend, for our experiments, we used the structured pruning strategy described in [19].…”

Towards Efficient Capsule Networks

“…As an effect, they minimize the cardinality of some i-th intermediate layer's output ∥x i ∥ 0 . Bragagnolo et al [20] showed that structured sparsity, despite removing significantly less parameters from the model, yields lower model's memory footprint and inference time. When pruning a network in a structured way, a simplification step which practically reduces the rank of the matrices is possible; on the other side, encoding unstructured sparse matrices lead to representation overheads [10].…”

The Rise of the Lottery Heroes: Why Zero-Shot Pruning is Hard

“…Can they be determined earlier, during a normal vanilla training? deployment time and making inference more efficient [7,8,9,10,11]. A recent work, the lottery ticket hypothesis [12], suggests that the fate of a parameter, namely whether it is useful for training (winner at the lottery of initialization) or if it can be removed from the architecture, is decided already at the initialization step.…”

“…return M 11: end procedure where u i is some generic update term. In principle, the parameters in W are not in the model, and for instance they should not be included in the computation anymore; however, we still need to encode that are missing, producing an overhead, as they are removed in an unstructured way [9]. 2 Limits.…”