Gradient Update using Layer-based Importance Ratio in Deep Neural Networks

Abstract

Our study delves into the widespread utilization of Deep Neural Networks (DNNs) across diverse domains, showcasing their notable successes. We evaluate the effectiveness of our approach on well-established network architectures such as AlexNet, VGG16, and ResNet50, employing widely used image classification datasets namely CIFAR-10 and CIFAR-100. Our findings reveal significant advancements in terms of both accuracy and compression ratio when compared to existing methodologies. Notably, our method achieves an impressive compression rate of 15× while maintaining consistent levels of accuracy across both datasets, marking a substantial leap forward in model efficiency without sacrificing performance.