Revisiting Internal Covariate Shift for Batch Normalization

Awais, Muhammad; Iqbal, Md. Tauhid Bin; Bae, Sung Ho

doi:10.1109/tnnls.2020.3026784

Cited by 63 publications

(26 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The strengthening of Ioffe and Szegedy's argument is obtained from a study by (Awais et al, 2021). A series of experiments in this study show that ICS reduction is a major factor in increasing the convergence of a deep learning network, not only by batch normalization but also by all other methods that contribute to ICS reduction.…”

Section: Plant Classificationsupporting

confidence: 56%

Parallelization of Concise Convolutional Neural Networks for Plant Classification

Sembiring¹,

Away²,

Arnia³

et al. 2023

J. Ecol. Eng.

View full text Add to dashboard Cite

Monitoring the agricultural field is the key to preventing the spread of disease and handling it quickly. The computer-based automatic monitoring system can meet the needs of large-scale and real-time monitoring. Plant classifiers that can work quickly in computer with limited resources are needed to realize this monitoring system. This study proposes convolutional neural network (CNN) architecture as a plant classifier based on leaf imagery. This architecture was built by parallelizing two concise CNN channels with different filter sizes using the addition operation. GoogleNet, SqueezeNet and MobileNetV2 were used to compare the performance of the proposed architecture. The classification performance of all these architectures was tested using the PlantVillage dataset which consists of 38 classes and 14 plant types. The experimental results indicated that the proposed architecture with a smaller number of parameters achieved nearly the same accuracy as the comparison architectures. In addition, the proposed architecture classified images 5.12 times faster than SqueezeNet, 8.23 times faster than GoogleNet, and 9.4 times faster than MobileNetV2. These findings suggest that when implemented in the agricultural field, the proposed architecture can be a reliable and faster plant classifier with fewer resources.

show abstract

Section: Plant Classificationsupporting

confidence: 56%

Parallelization of Concise Convolutional Neural Networks for Plant Classification

Sembiring¹,

Away²,

Arnia³

et al. 2023

J. Ecol. Eng.

View full text Add to dashboard Cite

show abstract

“…The number of possible implementations of model architectures is also large enough that finding the perfect working model for a problem cannot be achieved. Models can be improved by the addition or removal of layers and different activation functions within the layers, as well as by utilizing regularization techniques such as batch normalization [ 34 ] or dropout. The deep learning techniques employed in this research are based on many evaluations and modifications to the architectures, as well as an examination of the related literature.…”

Section: Resultsmentioning

confidence: 99%

Brain Age Prediction/Classification through Recurrent Deep Learning with Electroencephalogram Recordings of Seizure Subjects

Jusseaume

Valova

2022

Sensors

View full text Add to dashboard Cite

With modern population growth and an increase in the average lifespan, more patients are becoming afflicted with neurodegenerative diseases such as dementia and Alzheimer’s. Patients with a history of epilepsy, drug abuse, and mental health disorders such as depression have a larger risk of developing Alzheimer’s and other neurodegenerative diseases later in life. Utilizing recordings of patients’ brain waves obtained from the Temple University abnormal electroencephalogram (EEG) corpus, deep leaning long short-term memory neural networks are utilized to classify and predict patients’ brain ages. The proposed deep learning neural network model structure and brain wave-processing methodology leads to an accuracy of 90% in patients’ brain age classification across six age groups, with a mean absolute error value of 7 years for the brain age regression analysis. The achieved results demonstrate that the use of raw patient-sourced brain wave information leads to higher performance metrics than methods utilizing other brain wave-preprocessing methods and outperforms other deep learning models such as convolutional neural networks.

show abstract

“…It comprises of 7 Conv-ReLU-BN blocks, where Conv refers to the standard Convolution operation, ReLU refers to the Rectified Linear Activation layer and BN refers to Batch Normalization layer, used to keep the weights in within a threshold reducing internal covariate shift [45,46], thus stabilizing the training process. Dropout layers [47] were used prior to the last two dense layers, though it hasn't been represented in the Fig.…”

Section: Model Architecturementioning

confidence: 99%

Advances in Classifying the Stages of Diabetic Retinopathy Using Convolutional Neural Networks in Low Memory Edge Devices

Paul

2021

Preprint

View full text Add to dashboard Cite

Diabetic Retinopathy (DR) is a severe complication that may lead to retinal vascular damage and is one of the leading causes of vision impairment and blindness. DR broadly is classified into two stages - non-proliferative (NPDR), where there are almost no symptoms, except a few microaneurysms, and proliferative (PDR) involving a huge number of microaneurysms and hemorrhages, soft and hard exudates, neo-vascularization, macular ischemia or a combination of these, making it easier to detect. More specifically, DR is usually classified into five levels, labeled 0-4, from 0 indicating no DR to 4 which is most severe. This paper firstly presents a discussion on the risk factors of the disease, then surveys the recent literature on the topic followed by examining certain techniques which were found to be highly effective in improving the prognosis accuracy. Finally, a convolutional neural network model is proposed to detect all the stages of DR on a low-memory edge microcontroller. The model has a size of just 5.9 MB, accuracy and F1 score both of 94% and an inference speed of about 20 frames per second.

show abstract

Revisiting Internal Covariate Shift for Batch Normalization

Cited by 63 publications

References 8 publications

Parallelization of Concise Convolutional Neural Networks for Plant Classification

Parallelization of Concise Convolutional Neural Networks for Plant Classification

Brain Age Prediction/Classification through Recurrent Deep Learning with Electroencephalogram Recordings of Seizure Subjects

Advances in Classifying the Stages of Diabetic Retinopathy Using Convolutional Neural Networks in Low Memory Edge Devices

Contact Info

Product

Resources

About