MaxDropoutV2: An Improved Method to Drop Out Neurons in Convolutional Neural Networks

Santos, Claudio Filipi Gonçalves dos; Roder, Mateus; Passos, Leandro Aparecido; Papa, João Paulo

doi:10.1007/978-3-031-04881-4_22

Cited by 3 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is a regularization technique that stochastically adjusts the hidden unit activations for each training example to zero during training. Other stochastic model averaging techniques like stochastic pooling [24], drop-connect [25], and maxout networks [26] were influenced by dropout. The following figure (5) illustrates how the dropout layer affected the Low R50 and Low R18 input features.…”

Section: Dropout Layermentioning

confidence: 99%

Particulate Matter Levels Classification Using Modified and Combined ResNet Models with Low Features Extraction

Matloob¹,

Shakir²

2023

3C TIC

View full text Add to dashboard Cite

Smog is a serious environmental problem. It is an atmospheric pollutant that, if inhaled frequently, can lead to lung diseases such as asthma and bronchitis. One of the most dangerous air pollutants is particulate matter with a diameter of fewer than 2.5 micrometers (PM2.5), which may be breathed into the body and cause major health issues by introducing dangerous compounds deep into the lungs and bloodstream. In this research, a new convolutional neural network is proposed, by upgrading and parallelly stacking the two pre-trained models ResNet18 and ResNet50 to form a new modified-combined convolutional model (C-DCNN). Besides, we stacked another two columns of layers to extract the low features of ResNet18 and ResNet50 separately, to create finally four stacked columns of layers. The new model classifies images into different classes based on their PM2.5 concentration levels. To assess the suggested approach, an image augmentation is applied, then divided the images randomly (80% for the training progress,20% of the used training data for validation, and 20% for testing). The experimental results demonstrate that the proposed method increased the accuracy of level estimation with an accuracy increment equal to (6.25% at LR=0.

show abstract

Section: Dropout Layermentioning

confidence: 99%

Particulate Matter Levels Classification Using Modified and Combined ResNet Models with Low Features Extraction

Matloob¹,

Shakir²

2023

3C TIC

View full text Add to dashboard Cite

show abstract

“…An improvement on MaxDropout, called MaxDropout V2 [39] is proposed in the next work. It kept the concept of dropping neurons based on their values, however, it relies on a space correlation between the neurons.…”

Section: B Maxdropout V2mentioning

confidence: 99%

“…ResNet18 [17] 4.72 ± 0.21 22.46 ± 0.31 + Cutout [17] 3.99 ± 0.13 21.96 ± 0.24 + RandomErasing [19] 4.31 ± 0.07 24.03 ± 0.19 + Mixup [20] 4.2 21.1 + MM [32] 2.95 ± 0.04 20.34 ± 0.52 + TSLA [36] -21.45 ± 0.28 + TargetDrop [43] 4.41 21.37 + TargetDrop + Cutout [43] 3.67 21.25 + LocalDrop [29] 4.3 22.2 + MaxDropout [27] 4.66 ± 0.14 21.93 ± 0.07 + MaxDropoutV2 [39] 4.63 ± 0.03 21.92 ± 0.23 + MaxDropout + Cutout [27] 3 3.89 18.85 + TargetDrop [43] 3.68 -+ GradAug [28] 16.02 + Dropout + Cutout [17] 3.08 ± 0.16 18.41 ± 0.27 + Dropout + PBA [24] 2.58 ± 0.06 16.73 ± 0.15 + Dropout + RE [19] 3.08 ± 0.05 17.73 ± 0.15 + Dropout + BA + Cutout [15] 2.85 19.87 + ShakeDrop [31] 4.37 19.47 + Dropout + RE [19] 3.08 ± 0.05 17.73 ± 0.15 + Dropout + Mixup [20] 2.7 17.5 + Dropout + MM [32] 2.55 ± 0.02 18.04 ± 0.17 + Dropout + Fast AA [22] 2.7 17.3 + Dropout + RA [23] 2.7 16.7 + AutoDrop [34] 3.1 + AutoDrop + RE [34] 2.1 -+ MaxDropout [27] 3.84 18.81…”

Section: Software Ispmentioning

confidence: 99%

Avoiding Overfitting: new algorithms to improve generalization in Convolutional Neural Networks

Santos

Papa

2022

Anais Estendidos Do XXXV Conference on Graphics, Patterns and Images (SIBGRAPI Estendido 2022)

View full text Add to dashboard Cite

Deep Learning has achieved state-of-the-art results in several domains, such as im- age processing, natural language processing, and audio processing. To accomplish such results, it uses neural networks with several processing layers along with a massive amount of labeled information. One particular family of Deep Learning is the Convolutional Neural Networks (CNNs), which work using convolutional layers de- rived from the digital signal processing area, being very helpful to detect relevant features in unstructured data, such as audio and pictures. One way to improve results on CNN is to use regularization algorithms, which aim to make the training process harder but generate models that generalize better for inference when used in applications. The present work contributes to the area of regularization methods for CNNs, proposing more methods for use in different image processing tasks. This thesis presents a collection of works developed by the author during the research period, which were published or submitted until the present time, presenting: (i) a survey, listing recent regularization works and highlighting the solutions and problems of the area; (ii) a neuron dropping method to use in the tensors generated during CNNs training; (iii) a variation of the mentioned method, changing the dropping rules, targeting different features of the tensor; and (iv) a label regularization algorithm used in different image processing problems.

show abstract

Rethinking Regularization with Random Label Smoothing

dos Santos,

Papa

2024

Neural Process Lett

View full text Add to dashboard Cite

Regularization helps to improve machine learning techniques by penalizing the models during training. Such approaches act in either the input, internal, or output layers. Regarding the latter, label smoothing is widely used to introduce noise in the label vector, making learning more challenging. This work proposes a new label regularization method, Random Label Smoothing, that attributes random values to the labels while preserving their semantics during training. The idea is to change the entire label into fixed arbitrary values. Results show improvements in image classification and super-resolution tasks, outperforming state-of-the-art techniques for such purposes.

show abstract

MaxDropoutV2: An Improved Method to Drop Out Neurons in Convolutional Neural Networks

Cited by 3 publications

References 12 publications

Particulate Matter Levels Classification Using Modified and Combined ResNet Models with Low Features Extraction

Particulate Matter Levels Classification Using Modified and Combined ResNet Models with Low Features Extraction

Avoiding Overfitting: new algorithms to improve generalization in Convolutional Neural Networks

Rethinking Regularization with Random Label Smoothing

Contact Info

Product

Resources

About