Only Train Once: A One-Shot Neural Network Training And Pruning Framework

Abstract

Structured pruning is a commonly used technique in deploying deep neural networks (DNNs) onto resource-constrained devices. However, the existing pruning methods are usually heuristic, task-specified, and require an extra fine-tuning procedure. To overcome these limitations, we propose a framework that compresses DNNs into slimmer architectures with competitive performances and significant FLOPs reductions by Only-Train-Once (OTO). OTO contains two keys: (i) we partition the parameters of DNNs into zero-invariant groups, enabling us to prune zero groups without affecting the output; and (ii) to promote zero groups, we then formulate a structured-sparsity optimization problem and propose a novel optimization algorithm, Half-Space Stochastic Projected Gradient (HSPG), to solve it, which outperforms the standard proximal methods on group sparsity exploration and maintains comparable convergence. To demonstrate the effectiveness of OTO, we train and compress full models simultaneously from scratch without fine-tuning for inference speedup and parameter reduction, and achieve state-of-the-art results on VGG16 for CIFAR10, ResNet50 for CIFAR10/ImageNet and Bert for SQuAD.

Target Capacity Filter Pruning Method for Optimized Inference Time Based on YOLOv5 in Embedded Systems

“…Pruning is classified as either unstructured/weight pruning [14] or structured pruning [15]- [22] according to the method of determining the importance of parameters and removing them. Unstructured pruning [14] removes parameters by determining the importance of each parameter according to the saliency score of each algorithm.…”

“…Unstructured pruning also has the disadvantage of requiring specific library and hardware support for a sparse matrix. In contrast, structured pruning [15]- [22] judges the importance of network connections according to the saliency score of each algorithm based on larger units such as channels. Structured pruning does not have a sparse matrix so that an existing library can be used, and memory usage can be reduced without requiring additional hardware.…”

“…In particular, this made CNNs difficult to use in embedded devices such as mobiles, autonomous vehicles, and drones, where resources are limited. To solve this problem, pruning is being actively researched [14]- [22] in a variety of studies on lightweight deep learning networks.…”

“…After training the network to figure out the importance of the parameters, unimportant parameters are removed according to their saliency scores by each algorithm. Moreover, pruning is divided into unstructured pruning [14] and structured pruning [15]- [22]. Unstructured pruning [14] judges the importance of each parameter individually and removes unnecessary parameters to obtain a sparse convolution structure.…”

“…Unstructured pruning [14] judges the importance of each parameter individually and removes unnecessary parameters to obtain a sparse convolution structure. Structured pruning [15]- [22] removes unnecessary parameters by judging the importance of a bundle of several parameters. In this paper, by using structured pruning, YOLOv5 (an object detection network), is light-weighted to generate a network with a significantly reduced number of parameters and amount of computation.…”