Comparison of Various Learning Rate Scheduling Techniques on Convolutional Neural Network

Konar, Jinia; Khandelwal, Prerit; Tripathi, Rishabh

doi:10.1109/sceecs48394.2020.94

Cited by 39 publications

(17 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Thus, there exists a differentiation over water area marked by yellow in final output prediction map. Channels (55,56,57) cover lower bank and edges, however these are markedly different than the edges picked by (37,38,39). Channels (58,59,60) get activated at the right bank, marked by cyan in final output prediction map.…”

Section: Discussionmentioning

confidence: 81%

Bird-Area Water-Bodies Dataset (BAWD) and Predictive AI Model for Avian Botulism Outbreak (AVI-BoT)

Bhatia¹,

Devang²,

Singh³

2021

Preprint

View full text Add to dashboard Cite

Avian botulism caused by a bacterium, Clostridium botulinum, causes a paralytic disease in birds often leading to high fatality, and is usually diagnosed using molecular techniques. Diagnostic techniques for confirming the outbreak of Avian botulism include: Mouse Bioassay, ELISA, PCR (based on post-mortem), all of which are time-consuming, laborious and require invasive sample collection from affected sites or dead birds. All these in-vitro techniques are non-preventive and their processing time may further contribute to the bacterial growth in the affected regions. In this study, we build a first-ever multi-spectral, remote-sensing imagery based global Bird-Area Water-bodies Dataset (BAWD) (i.e. fused satellite images of warm-water lakes/marshy-lands or similar waterbody sites that are important for avian fauna) backed by on-ground reporting evidence of outbreaks. In the current version, BAWD consists of multi-spectral satellite images, covering a total ground area of 904 sq.km from two open source satellite projects (Sentinel and Landsat). BAWD consists of 17 topographically diverse global sites spanning across 4 continents, where locations were monitored over a time-span of 3 years (2016-2020). Using BAWD and state-of-the-art deep-learning techniques we propose a first-ever Artificial Intelligence based (AI) model to predict potential outbreak of Avian botulism called AVI-BoT (Aerosol Visible, Infra-red (NIR/SWIR) and Bands of Thermal). AVI-BoT uses fused multi-spectral satellite images of water-bodies (10-bands) as input to generate a spatial prediction map depicting probability of potential Avian botulism outbreaks. We also train and investigate a simpler (5-band) Causative-Factor model (based on prominent physiological factors reported in literature as conducive for outbreak) to predict Avian botulism. Using AVI-BoT, we achieve a training accuracy of 0.94 and validation accuracy of 0.96 on BAWD, far superior in comparison to our model based on causative factors. For detailed feature-space analysis and explain-ability of the proposed 10-band AVI-BoT model, we further train four additional spectral models on BAWD using: (i) Aerosol, (ii) Visible (3-bands), (iii) IR (3-bands), and (iv) Thermal (3bands). Further, we analyze using AVI-BoT, two recent and highly complex outbreak case-studies: (i) Sambhar Lake (India, November 2019), and (ii) Klamath National Wildlife Refuge (U.S.A., October 2020). The model is found to closely converge with ground-reporting observations at both these sites with a fine-grain spatio-temporal prediction capability. The proposed technique presents a scale-able, low-cost, non-invasive methodology for continuous monitoring of bird-habitats against botulism outbreaks with the potential of saving valuable fauna lives.

show abstract

Section: Discussionmentioning

confidence: 81%

Bird-Area Water-Bodies Dataset (BAWD) and Predictive AI Model for Avian Botulism Outbreak (AVI-BoT)

Bhatia¹,

Devang²,

Singh³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In deep learning, the learning rate is one of the hyperparameters that decides the step size at each time the model progresses to a minimum loss function. Hence, it is crucial to optimize the learning rate properly; otherwise, the model may converge slowly with too small learning rate or diverge from the optimal error points with large learning rate values [55]. is study sets three values (1e − 2, 1e − 3, or 1e − 4) to choose using Keras Tuner.…”

Section: Learning Ratementioning

confidence: 99%

Short-Term Daily Univariate Streamflow Forecasting Using Deep Learning Models

Wegayehu

Muluneh

2022

Advances in Meteorology

View full text Add to dashboard Cite

Hydrological forecasting is one of the key research areas in hydrology. Innovative forecasting tools will reform water resources management systems, flood early warning mechanisms, and agricultural and hydropower management schemes. Hence, in this study, we compared Stacked Long Short-Term Memory (S-LSTM), Bidirectional Long Short-Term Memory (Bi-LSTM), and Gated Recurrent Unit (GRU) with the classical Multilayer Perceptron (MLP) network for one-step daily streamflow forecasting. The analysis used daily time series data collected from Borkena (in Awash river basin) and Gummera (in Abay river basin) streamflow stations. All data sets passed through rigorous quality control processes, and null values were filled using linear interpolation. A partial autocorrelation was also applied to select the appropriate time lag for input series generation. Then, the data is split into training and testing datasets using a ratio of 80 : 20, respectively. Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), and coefficient of determination (R2) were used to evaluate the performance of the proposed models. Finally, the findings are summarized in model variability, lag time variability, and time series characteristic themes. As a result, time series characteristics (climatic variability) had a more significant impact on streamflow forecasting performance than input lagged time steps and deep learning model architecture variations. Thus, Borkena’s river catchment forecasting result is more accurate than Gummera’s catchment forecasting result, with RMSE, MAE, MAPE, and R2 values ranging between (0.81 to 1.53, 0.29 to 0.96, 0.16 to 1.72, 0.96 to 0.99) and (17.43 to 17.99, 7.76 to 10.54, 0.16 to 1.03, 0.89 to 0.90) for both catchments, respectively. Although the performance is dependent on lag time variations, MLP and GRU outperform S-LSTM and Bi-LSTM on a nearly equal basis.

show abstract

“…Choosing the learning rate may be difficult, since a value that is too small can lead to a lengthy training procedure with significant training error, whereas a value too big can lead to learning a sub-optimal set of weights too quickly (without reaching the local minima) or an unstable training process [51]. To reduce the learning rate, ReduceLROnPlateau was used [52]. When learning becomes static, models frequently benefit from reducing the learning rate by a factor of 2-10.…”

Section: Learning Rate Optimisationmentioning

confidence: 99%

Deep Transfer Learning for Land Use and Land Cover Classification: A Comparative Study

Naushad¹,

Kaur

Ghaderpour

2021

Sensors

119

View full text Add to dashboard Cite

Efficiently implementing remote sensing image classification with high spatial resolution imagery can provide significant value in land use and land cover (LULC) classification. The new advances in remote sensing and deep learning technologies have facilitated the extraction of spatiotemporal information for LULC classification. Moreover, diverse disciplines of science, including remote sensing, have utilised tremendous improvements in image classification involving convolutional neural networks (CNNs) with transfer learning. In this study, instead of training CNNs from scratch, the transfer learning was applied to fine-tune pre-trained networks Visual Geometry Group (VGG16) and Wide Residual Networks (WRNs), by replacing the final layers with additional layers, for LULC classification using the red–green–blue version of the EuroSAT dataset. Moreover, the performance and computational time are compared and optimised with techniques such as early stopping, gradient clipping, adaptive learning rates, and data augmentation. The proposed approaches have addressed the limited-data problem, and very good accuracies were achieved. The results show that the proposed method based on WRNs outperformed the previous best results in terms of computational efficiency and accuracy, by achieving 99.17%.

show abstract

Comparison of Various Learning Rate Scheduling Techniques on Convolutional Neural Network

Cited by 39 publications

References 10 publications

Bird-Area Water-Bodies Dataset (BAWD) and Predictive AI Model for Avian Botulism Outbreak (AVI-BoT)

Bird-Area Water-Bodies Dataset (BAWD) and Predictive AI Model for Avian Botulism Outbreak (AVI-BoT)

Short-Term Daily Univariate Streamflow Forecasting Using Deep Learning Models

Deep Transfer Learning for Land Use and Land Cover Classification: A Comparative Study

Contact Info

Product

Resources

About