Skin Cancer Classification based on Cosine Cyclical Learning Rate with Deep Learning

Nie, Yali; Carratù, Marco; O’Nils, Mattias; Sommella, Paolo; Ugwiri, Moise Avoci; Lundgren, Jan

doi:10.1109/i2mtc48687.2022.9806568

Cited by 6 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The prediction outputs in each cycle are then combined as a final prediction. In application, Nie et al [55] used cosine annealing LR in the DL model to classify skin cancer. The authors demonstrated their proposed deep model using the annealing method and compared the result with a fixed LR schedule.…”

Section: Literature Reviewmentioning

confidence: 99%

ps-CALR: Periodic-Shift Cosine Annealing Learning Rate for Deep Neural Networks

Johnson,

Xinying,

Khaw

et al. 2023

IEEE Access

View full text Add to dashboard Cite

There are continued efforts to build on the performance of deep learning (DL) models in various fields of application. Developing new DL models continues to open unprecedented opportunities in diverse application areas despite the enormous resources required. Generally, the learning mechanism of DL models depends on the term "cost function" (CF) or "loss function" (LF), and DL models require varied hyperparameter settings and, precisely, parameters that can help the model to continually minimize the cost function until faster convergence, with better generalization over the data in the loss landscape, is assumed. The learning rate (LR) update seeks to find the optimal solution for DL models through relative cost function minimization. Therefore, selecting the appropriate LR is essential to the performance of DL models. The existing cosine annealing LR, despite its demonstration for fast model convergence, lacks complete loss landscape exploration of the flat minima, hence limiting its ability to model better generalization. To address this, the paper proposes a period-shift Cosine Annealing Learning Rate with warm-up epochs (ps-CALR) to perturb the LR update. Six different publicly available datasets were used to benchmark the proposed LR method by experimenting with custom DL (multilayer perceptron and convolutional neural networks) and pre-trained DL models. The proposed ps-CARL enhances model generalization and convergence, pushing the solution to notably better performance compared with fixed LR and the existing cosine annealing method.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

ps-CALR: Periodic-Shift Cosine Annealing Learning Rate for Deep Neural Networks

Johnson,

Xinying,

Khaw

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Moreover, adaptive LR methods have been established for improved learning outcomes [12][13][14]. Cyclic learning rate policies have leveraged the benefits of the warm restart, which periodically adjusts the LR to enhance performance [15][16][17]. Warm restart refers to the occasional increment of the learning rate during training, typically used to evade local minima within the context of a cyclic learning rate policy.…”

Section: Introductionmentioning

confidence: 99%

Enhanced Classification of Sugarcane Diseases Through a Modified Learning Rate Policy in Deep Learning

Daphal,

Koli

2024

View full text Add to dashboard Cite

Motivated by the agriculture-centric economy of India, and specifically the challenges experienced in the sugarcane sector due to reduced yields from diseases including rust, red rot, yellow leaf, and mosaic, this study aims to harness effective deep-learning technologies for improved plant disease monitoring. The challenge of mitigating over-fitting, particularly when dealing with small datasets, is addressed through hyper-parameter tuning. In this study, we introduce an innovative modification to the learning rate decay policy, tested on a uniquely constructed small-sized database of sugarcane leaf images. This database encompasses five classes: healthy, rust, red rot, yellow leaf, and mosaic. To evaluate the effectiveness of the proposed learning rate policy, it was compared against multiple benchmark datasets and found to surpass established methods in performance metrics. This study introduces an additional exponential component into the learning rate policy to facilitate model convergence within the same number of epochs, thereby enhancing its performance over step, exponential, cosine, and exponential sine methods. A marginal improvement in scores was observed with the integration of the proposed learning rate policy and MobileNet-V2 as the backbone architecture. Remarkably, the MNIST dataset achieved a score of 99.9%, CIFAR-10 scored 92%, and the newly introduced database secured a score of 89%. These results underscore the efficacy of the proposed approach in enhancing the classification of sugarcane diseases.

show abstract