Model Compression Using Progressive Channel Pruning

Guo, Jinyang; Zhang, Weichen; Ouyang, Wanli; Xu, Dong

doi:10.1109/tcsvt.2020.2996231

Cited by 54 publications

(12 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Various retraining-based methods are proposed in the literature (Ström 1997;Han et al 2015;Molchanov, Ashukha, and Vetrov 2017;Frankle and Carbin 2019;Renda, Frankle, and Carbin 2020;Narang et al 2017;Guo et al 2020;Guo, Xu, and Ouyang 2023;Huang et al 2023) to improve the accuracy. Although some of them may show promise, they often require extensive training or tuning of hyper-parameters, leading to inefficiency and slow convergence.…”

Section: Related Workmentioning

confidence: 99%

Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes

Gong,

Yong,

Wang

et al. 2024

AAAI

View full text Add to dashboard Cite

Neural network sparsity has attracted many research interests due to its similarity to biological schemes and high energy efficiency. However, existing methods depend on long-time training or fine-tuning, which prevents large-scale applications. Recently, some works focusing on post-training sparsity (PTS) have emerged. They get rid of the high training cost but usually suffer from distinct accuracy degradation due to neglect of the reasonable sparsity rate at each layer. Previous methods for finding sparsity rates mainly focus on the training-aware scenario, which usually fails to converge stably under the PTS setting with limited data and much less training cost. In this paper, we propose a fast and controllable post-training sparsity (FCPTS) framework. By incorporating a differentiable bridge function and a controllable optimization objective, our method allows for rapid and accurate sparsity allocation learning in minutes, with the added assurance of convergence to a predetermined global sparsity rate. Equipped with these techniques, we can surpass the state-of-the-art methods by a large margin, e.g., over 30\% improvement for ResNet-50 on ImageNet under the sparsity rate of 80\%. Our plug-and-play code and supplementary materials are open-sourced at https://github.com/ModelTC/FCPTS.

show abstract

Section: Related Workmentioning

confidence: 99%

Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes

Gong,

Yong,

Wang

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Model compression can reduce computation cost and storage consumption, meanwhile maintaining the loss of performance within a small scale. Guo et al [27] proposed PCP (Progressive Channel Pruning), a model pruning method with automatically determined network structure, to prune channels iteratively. Recently, attention mechanisms have been inserted into the model pruning method for improving performance.…”

Section: B Comparison Experimentsmentioning

confidence: 99%

CroApp: A CNN-Based Resource Optimization Approach in Edge Computing Environment

Jia

Liu

Dou

et al. 2022

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

With the emergence of various CNN-based applications and the rapid growth of CNN model scale, the resource-constricted end devices can hardly deploy CNN-based applications. Current work optimize the CNN model on edge servers and deploy the optimized model on devices in edge computing environment. However, most of them only optimize the resource consumption within or across models solely, whereas neglecting the other side. In this paper, we propose a novel CNNbased Resource Optimization APProach (CroApp) that not only optimizes the resource consumption within the CNN model but also pays attention to resource optimization across the applications. Specifically, we adopt model compression as the "innermodel" optimization method, as well as computation sharing as the "inter-model" optimization method. Firstly, during "innermodel" optimization, CroApp prunes unnecessary parameters within model on edge servers to reduce the scale of the model. Then, during "inter-model" optimization, CroApp trains a set of shareable models based on the pruned model and sends these shareable models to end devices. Finally, CroApp adaptively adjusts the shared models to reduce resource consumption. The experimental results show that CroApp outperforms the stateof-the-art approaches in terms of resource reduction, scalability, and application performance.

show abstract

“…For a given neural network, model compression may be another effective method to reduce the parameters and computation. Researchers have proposed some model pruning techniques to reduce the model size with an allowed accuracy range by removing useless channels for easier acceleration or unimportant connections between neurons [36,37]. In deep neural networks, weights are usually stored in the form of 32-bit floating-point numbers [38], and the model quantization compresses the original network by reducing the number of bits required to represent each weight [28,39].…”

Section: Related Workmentioning

confidence: 99%

A cell phone app for facial acne severity assessment

et al. 2022

View full text Add to dashboard Cite

Acne vulgaris, the most common skin disease, can cause substantial economic and psychological impacts to the people it affects, and its accurate grading plays a crucial role in the treatment of patients. In this paper, we firstly proposed an acne grading criterion that considers lesion classifications and a metric for producing accurate severity ratings. Due to similar appearance of acne lesions with comparable severities and difficult-to-count lesions, severity assessment is a challenging task. We cropped facial skin images of several lesion patches and then addressed the acne lesion with a lightweight acne regular network (Acne-RegNet). Acne-RegNet was built by using a median filter and histogram equalization to improve image quality, a channel attention mechanism to boost the representational power of network, a region-based focal loss to handle classification imbalances and a model pruning and feature-based knowledge distillation to reduce model size. After the application of Acne-RegNet, the severity score is calculated, and the acne grading is further optimized by the metadata of the patients. The entire acne assessment procedure was deployed to a mobile device, and a phone app was designed. Compared with state-of-the-art lightweight models, the proposed Acne-RegNet significantly improves the accuracy of lesion classifications. The acne app demonstrated promising results in severity assessments (accuracy: 94.56%) and showed a dermatologist-level diagnosis on the internal clinical dataset.The proposed acne app could be a useful adjunct to assess acne severity in clinical practice and it enables anyone with a smartphone to immediately assess acne, anywhere and anytime.

show abstract

Model Compression Using Progressive Channel Pruning

Cited by 54 publications

References 22 publications

Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes

Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes

CroApp: A CNN-Based Resource Optimization Approach in Edge Computing Environment

A cell phone app for facial acne severity assessment

Contact Info

Product

Resources

About