Ensemble Knowledge Guided Sub-network Search and Fine-Tuning for Filter Pruning

Lee, Seunghyun; Song, Byung Cheol

doi:10.1007/978-3-031-20083-0_34

Cited by 5 publications

(1 citation statement)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Clustering-based pruning Traditional techniques (Duggal et al 2019;Lee et al 2020) exploit differences in filter distributions across layers (He et al 2019) to identify similarities between channels. These methods group similar filters or representations, mapping them to a latent space, and retain only the essential ones.…”

Section: Related Workmentioning

confidence: 99%

REPrune: Channel Pruning via Kernel Representative Selection

Park,

Kim,

Park

et al. 2024

AAAI

View full text Add to dashboard Cite

Channel pruning is widely accepted to accelerate modern convolutional neural networks (CNNs). The resulting pruned model benefits from its immediate deployment on general-purpose software and hardware resources. However, its large pruning granularity, specifically at the unit of a convolution filter, often leads to undesirable accuracy drops due to the inflexibility of deciding how and where to introduce sparsity to the CNNs. In this paper, we propose REPrune, a novel channel pruning technique that emulates kernel pruning, fully exploiting the finer but structured granularity. REPrune identifies similar kernels within each channel using agglomerative clustering. Then, it selects filters that maximize the incorporation of kernel representatives while optimizing the maximum cluster coverage problem. By integrating with a simultaneous training-pruning paradigm, REPrune promotes efficient, progressive pruning throughout training CNNs, avoiding the conventional train-prune-finetune sequence. Experimental results highlight that REPrune performs better in computer vision tasks than existing methods, effectively achieving a balance between acceleration ratio and performance retention.

show abstract

Section: Related Workmentioning

confidence: 99%