Methodologies of Compressing a Stable Performance Convolutional Neural Networks in Image Classification

Al-Hami, Mo'taz; Pietroń, Marcin; Casas, R.A.; Wielgosz, Maciej

doi:10.1007/s11063-019-10076-y

Cited by 19 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Works in [1,7,20,26,28,43,50] concentrate on structural pruning, which involves pruning entire channels in filters by means of different techniques. Non-structural pruning can be carried out on pretrained models without retraining as in [37,2], which results in increased efficiency, or with retraining, as in lottery tickets search [15], movement pruning [41,18], and variational dropout [33]. Other efficient pruning methods include genetic-based approaches as in [45], which adopts constant masks and defines its own crossover and mutation operators, and reinforcement techniques as in [17].…”

Section: Related Workmentioning

confidence: 99%

Ada-QPacknet – Multi-Task Forget-Free Continual Learning with Quantization Driven Adaptive Pruning

Pietron,

Zurek,

Faber

et al. 2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Continual learning (CL) is a challenging machine learning setting that is attracting the interest of an increasing number of researchers. Among recent CL works, architectural strategies appear particularly promising due to their potential to expand and adapt the model architecture as new tasks are presented. However, existing solutions do not efficiently exploit model sparsity due to the adoption of constant pruning ratios. Moreover, current approaches exhibit a tendency to quickly saturate model capacity since the number of weights is limited and each weight is restricted to a single value. In this paper, we propose Ada-QPacknet, a novel architectural CL method that resorts to adaptive pruning and quantization. These two features allow our model to overcome the two crucial issues of effective exploitation of model sparsity and efficient use of model capacity. Specifically, adaptive pruning restores model capacity by reducing the number of weights assigned to each task to a smaller subset of weights that preserves the performance of the full set, allowing other weights to be used for future tasks. Adaptive quantization separates each weight into multiple components with adaptively reduced bit-width, allowing a single weight to solve more than one task without significant performance drops, leading to improved exploitation of model capacity. Experimental results on benchmark CL scenarios show that our proposed method achieves better results in terms of accuracy than existing rehearsal, regularization, and architectural CL strategies. Moreover, our method significantly outperforms forget-free competitors in terms of efficient exploitation of model capacity.

show abstract

Section: Related Workmentioning

confidence: 99%

Ada-QPacknet – Multi-Task Forget-Free Continual Learning with Quantization Driven Adaptive Pruning

Pietron,

Zurek,

Faber

et al. 2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

show abstract

“…The recent breakthrough in DCNNs has led to improvements in the accuracy of both vision and auditory systems, such as image recognition, object detection and speech recognition [2][3][4][5][6][7]. DCNNs are generally composed of several cascades of basic layers.…”

Section: Algorithmmentioning

confidence: 99%

“…The non-linear neuron is called activation function, which has the ability to solve non-linear problems. Rectified Linear Unit (ReLU) function is the most common one, as shown in equation (2). Paper [26] demonstrates that training convergence can be much faster in the network with ReLU neurons than that with other neurons.…”

Section: Algorithmmentioning

confidence: 99%

“…In recent years, there has been an increasing interest in DCNNs, due to their high accuracy and superior performance in tasks including image classification [2,3], object detection [4,5], and stereo vision [6,7]. This technique has also become a successful algorithm in the region of AI, for example, in AlphaGo algorithm, and it is a strong candidate for many computer vision algorithms.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

AlphaGo Policy Network: A DCNN Accelerator on FPGA

Zhu

Gao

et al. 2020

IEEE Access

View full text Add to dashboard Cite

The game of GO has long been regarded as the most challenging game for artificial intelligence because of its enormous search space and the difficulty of evaluating its board positions. In early 2016, the defeat of Lee Sedol by AlphaGo became the milestone of artificial intelligence. AlphaGo's success lies in that it efficiently combines policy and value networks with Monte Carlo tree search (MCTS). And these deep convolutional neural networks (DCNNs) are trained by the combination of supervised learning and reinforcement learning. However, large convolution operations are computationally-intensive and typically require a powerful computing platform, for example, a graphics processing unit (GPU). Therefore, it is challenging to apply DCCNs in resource-limited embedded systems. Field programmable gate array (FPGA) is proposed to be an appropriate solution to implement real-time DCCNs models. However, the limited bandwidth and on-chip memory storage are the bottlenecks for DCCNs acceleration. In this paper, an AlphaGo Policy Network is designed, and efficient hardware architectures are proposed to accelerate the DCCN model. The accelerator can be fit into different FPGAs, providing the balancing between processing speed and hardware resources. As an example, the AlphaGo Policy Network is implemented on Xilinx design suite VCU118, and the results show that our implementation achieved a performance of 3036.32 GOPS and achieved up to 56x speedup compared to CPU and 22.4x speedup compared to GPU.

show abstract

“…In recent years, with the advancement of machine learning, particularly deep learning, it has been a research hotspot that the robot avoids obstacles by self-learning (Z He, et al, 2021) [2]. An end-to-end learning strategy, also known as deep learning, is a mapping relationship between inputs and output that is achieved via the use of a deep learning network.…”

Section: Introductionmentioning

confidence: 99%

Obstacle avoidance and scene switching recognition for autonomous underwater vehicles based on convolutional neural networks

Zhang¹,

Ge²

2022

International Symposium on Robotics, Artificial Intelligence, and Information Engineering (RAIIE 2022)

View full text Add to dashboard Cite

As one of the important means for human beings to understand and develop the ocean, the application of underwater robots in marine science and engineering has attracted more and more attention. Underwater robots are divided into two categories, one is remotely operated underwater vehicle (ROV) and the other is autonomous underwater vehicle (AUV). Autonomous submersibles do not carry cables. Its energy is installed on the robot. Its task execution process is controlled bycomputer. Complete the task independently according to the robot program. Because there is no cable constraint, further analysis data can be collected after returning. AUV has the characteristics of autonomous navigation and large-scale observation. This paper presents a method based on convolution neural network, which realizes the effective perception and recognition of underwater environment. Through the ROS operating system, we integrate the control and environmental perception of the underwater robot, and design an underwater robot that can avoid obstacles and recognize autonomously.

show abstract

Methodologies of Compressing a Stable Performance Convolutional Neural Networks in Image Classification

Cited by 19 publications

References 17 publications

Ada-QPacknet – Multi-Task Forget-Free Continual Learning with Quantization Driven Adaptive Pruning

Ada-QPacknet – Multi-Task Forget-Free Continual Learning with Quantization Driven Adaptive Pruning

AlphaGo Policy Network: A DCNN Accelerator on FPGA

Obstacle avoidance and scene switching recognition for autonomous underwater vehicles based on convolutional neural networks

Contact Info

Product

Resources

About