Versatile Communication Optimization for Deep Learning by Modularized Parameter Server

Wu, Po-Yen; Liu, Pangfeng; Wu, Jan‐Jan

doi:10.1109/bigdata.2018.8622358

Cited by 1 publication

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among them, quantization (Han et al, 2015) is the most direct form of expression. (Wu et al, 2018) achieves compression through weight-sharing by k-means clustering on the weights. However, since the shared parameters need to be restored to their original locations, there are no runtime memory savings.…”

Section: Parameter Sharingmentioning

confidence: 99%

“…In related research, many compression methods for neural network have been proposed successively, including parameter pruning (Hu et al, 2016;Pan et al, 2020;Sui et al, 2021), parameter sharing (Wu et al, 2018), low-rank decomposition (Swaminathan et al, 2020) and knowledge distillation (Li et al, 2020;Prakosa et al, 2021), etc. But most of these methods have certain limitations, such as not being applicable to large neural network models, being applicable only to classification tasks, or some parameter settings depending on empirical and so on (Neill, 2020;Rongrong et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Optimization Method For Pruning Rates of Each Layer in CNN Based on the GA-SMSM

Dong

Song

2023

Preprint

View full text Add to dashboard Cite

Parameter pruning is one of the main methods for compressing CNN models. It can reduce the redundant parameters, the complexity of time and space, and calculation resources of the network while ensuring the minimal loss of the network's performance. Currently, most existing parameter pruning methods have equal pruning rates in all layers. Unlike previous methods, this paper focuses on the optimal combination of each layer’s pruning rates for a given pruning rate of the whole model. First, we use the genetic algorithm to obtain the pruning rate for each layer, where the pruning rate for each layer is not necessarily the same, but the average of the pruning rates is no more than the given pruning rate. Then, we order the similarity of filters to determine the pruning objects, using the sum of matrix similarity based on cosine similarity. We use VGGNet and ResNet for experimental validation on CIFAR10 and ImageNet ILSVRC2012.The results show that the accuracy loss of the pruned model using GA-SMSM is lower than that of previous methods.

show abstract

Section: Parameter Sharingmentioning

confidence: 99%