EvoPruneDeepTL: An evolutionary pruning model for transfer learning based deep neural networks

Poyatos, Javier; Molina, Daniel; Martinez, Aritz D.; Ser, Javier Del; Herrera, Francisco

doi:10.1016/j.neunet.2022.10.011

Cited by 30 publications

(6 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To deploy the model to embedded devices in the future, the next step is to further optimize the SUS-YOLOv5 model and implement compression of the model using lightweight techniques, such as pruning and knowledge distillation (Poyatos et al, 2022). Based on the detection objects, the genetic algorithm (Wang et al, 2022), Bayesian optimization algorithm (Lan et al, 2022), and particle swarm optimization algorithm may be used to determine hyperparameters of SUS-YOLOv5.…”

Section: Discussion and Future Workmentioning

confidence: 99%

Multi-object detection and behavior tracking of sea cucumbers with skin ulceration syndrome based on deep learning

Ge,

Xuan,

Lou

et al. 2024

Front. Mar. Sci.

View full text Add to dashboard Cite

Skin ulceration syndrome of sea cucumbers is one of the most serious diseases in intensive aquaculture, and it is the most effective way of preventing the spread of this disease to detect the abnormal behavior of sea cucumbers in time and take corresponding measures. However, the detection and tracking of multi-object is a hard problem in sea cucumber behavior analysis. To solve this problem, this paper first proposes a novel one-stage algorithm SUS-YOLOv5 for multi-object detection and tracking of sea cucumbers. The proposed SUS-YOLOv5 optimizes the maximum suppression algorithm in the overlapping region of the object detection box. Next, the SE-BiFPN feature fusion structure is proposed to enhance the transmission efficiency of feature information between deep and shallow layers of the network. Then, a MO-Tracking algorithm is proposed integrated with DeepSORT to achieve real-time multi-object tracking. Experimental results show that the mAP@0.5 and mAP@0.5:0.95 of the proposed object detector reach 95.40% and 83.80%, respectively, which are 3.30% and 4.10% higher than the original YOLOv5s. Compared with the traditional SSD, YOLOv3, and YOLOv4, the mAP of SUS-YOLOv5 is improved by 5.49%, 1.57%, and 3.76%, respectively. This research can realize the multi-object detection and tracking, which lays the foundation for the prediction of skin ulceration syndrome in sea cucumbers and has a certain practical application value for improving the intelligence level of aquaculture.

show abstract

Section: Discussion and Future Workmentioning

confidence: 99%

Multi-object detection and behavior tracking of sea cucumbers with skin ulceration syndrome based on deep learning

Ge,

Xuan,

Lou

et al. 2024

Front. Mar. Sci.

View full text Add to dashboard Cite

show abstract

“…These severely influence plant health, structure quality, production, quantity, and the economy. One of the highly complex tasks regarding plant protection is the timely identification of plant symptoms, pests , and diseases [ 4 ]. Traditional approaches used in underdeveloped or developing nations are through human eye inspection, which is inaccurate, tedious, and time-consuming.…”

Section: Introductionmentioning

confidence: 99%

“…While a considerable number of studies availed some plant disease classification and detection models, there are notable deficiencies in these studies [ 4 , 15 , 17 , 20 ], including training on limited dataset size leading to model overfitting and generalization complexity to diverse environments. Training models under controlled backgrounds and environmental conditions, in contrast to the natural setting that makes these models impractical in the natural environment, the accuracy and robustness of models.…”

Section: Introductionmentioning

confidence: 99%

Using transfer learning-based plant disease classification and detection for sustainable agriculture

Shafik,

Tufail,

De Silva Liyanage

et al. 2024

BMC Plant Biol

View full text Add to dashboard Cite

Subsistence farmers and global food security depend on sufficient food production, which aligns with the UN's “Zero Hunger,” “Climate Action,” and “Responsible Consumption and Production” sustainable development goals. In addition to already available methods for early disease detection and classification facing overfitting and fine feature extraction complexities during the training process, how early signs of green attacks can be identified or classified remains uncertain. Most pests and disease symptoms are seen in plant leaves and fruits, yet their diagnosis by experts in the laboratory is expensive, tedious, labor-intensive, and time-consuming. Notably, how plant pests and diseases can be appropriately detected and timely prevented is a hotspot paradigm in smart, sustainable agriculture remains unknown. In recent years, deep transfer learning has demonstrated tremendous advances in the recognition accuracy of object detection and image classification systems since these frameworks utilize previously acquired knowledge to solve similar problems more effectively and quickly. Therefore, in this research, we introduce two plant disease detection (PDDNet) models of early fusion (AE) and the lead voting ensemble (LVE) integrated with nine pre-trained convolutional neural networks (CNNs) and fine-tuned by deep feature extraction for efficient plant disease identification and classification. The experiments were carried out on 15 classes of the popular PlantVillage dataset, which has 54,305 image samples of different plant disease species in 38 categories. Hyperparameter fine-tuning was done with popular pre-trained models, including DenseNet201, ResNet101, ResNet50, GoogleNet, AlexNet, ResNet18, EfficientNetB7, NASNetMobile, and ConvNeXtSmall. We test these CNNs on the stated plant disease detection and classification problem, both independently and as part of an ensemble. In the final phase, a logistic regression (LR) classifier is utilized to determine the performance of various CNN model combinations. A comparative analysis was also performed on classifiers, deep learning, the proposed model, and similar state-of-the-art studies. The experiments demonstrated that PDDNet-AE and PDDNet-LVE achieved 96.74% and 97.79%, respectively, compared to current CNNs when tested on several plant diseases, depicting its exceptional robustness and generalization capabilities and mitigating current concerns in plant disease detection and classification.

show abstract

“…Chen et al [27] propose a deep neural model compression technique that achieves competitive performance with state-of-the-art methods and leverages the mutual information between the feature maps and the model output to eradicate redundant network layers. Poyatos et al [28] explore a novel evolutionary pruning model for a transfer learning-based deep neural network, in which the last fully connected layers are replaced by sparse layers optimized by a genetic algorithm, thereby performing optimized pruning or feature selection over the densely connected part of the neural network.…”

Section: Introductionmentioning

confidence: 99%

“…Poyatos et al. [ 28 ] explore a novel evolutionary pruning model for a transfer learning‐based deep neural network, in which the last fully connected layers are replaced by sparse layers optimized by a genetic algorithm, thereby performing optimized pruning or feature selection over the densely connected part of the neural network.…”

Section: Introductionmentioning

confidence: 99%

A Novel Learning Approach to Remove Oscillations in First‐Order Takagi–Sugeno Fuzzy System: Gradient Descent‐Based Neuro‐Fuzzy Algorithm Using Smoothing Group Lasso Regularization

Liu,

Wang,

Liu

et al. 2023

Advcd Theory and Sims

View full text Add to dashboard Cite

As a universal approximator, the first order Takagi–Sugeno fuzzy system possesses the capability to approximate widespread nonlinear systems through a group of IF THEN fuzzy rules. Although group lasso regularization has the advantage of inducing group sparsity and handling variable selection issues, it can lead to numerical oscillations and theoretical challenges in calculating the gradient at the origin when employed directly during training. The paper addresses the aforementioned obstacle by invoking a smoothing function to approximate group lasso regularization. On this basis, a gradient‐based neuro fuzzy learning algorithm with smoothing group lasso regularization for the first order Takagi–Sugeno fuzzy system is proposed. The convergence of the proposed algorithm is rigorously proved under gentle conditions. In addition, experimental outcomes acquired on two approximations and two classification simulations demonstrate that the proposed algorithm outperforms the algorithm with original group lasso regularization and L2 regularization in terms of error, pruned neurons, and accuracy. This is particularly evident in significant advancements in pruned neurons due to group sparsity. In comparison to the algorithm with L2 regularization, the proposed algorithm exhibits improvements of 6.3, 5.3, and 142.6 in pruned neurons during sin function, Gabor function, and Sonar benchmark dataset simulations, respectively.

show abstract

EvoPruneDeepTL: An evolutionary pruning model for transfer learning based deep neural networks

Cited by 30 publications

References 55 publications

Multi-object detection and behavior tracking of sea cucumbers with skin ulceration syndrome based on deep learning

Multi-object detection and behavior tracking of sea cucumbers with skin ulceration syndrome based on deep learning

Using transfer learning-based plant disease classification and detection for sustainable agriculture

A Novel Learning Approach to Remove Oscillations in First‐Order Takagi–Sugeno Fuzzy System: Gradient Descent‐Based Neuro‐Fuzzy Algorithm Using Smoothing Group Lasso Regularization

Contact Info

Product

Resources

About