Comparison of the stochastic gradient descent based optimization techniques

Yazan, Ersan; Talu, Muhammed Fatih

doi:10.1109/idap.2017.8090299

Cited by 73 publications

(29 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Adadelta optimization was used, and it is widely used in the field of deep learning among gradient descent-based optimization methods [ 42 ]. Adadelta is an extension of Adagrad with the aim of minimizing the intense and monotonically declining learning rate [ 43 , 44 ], and it limits the window of the accumulated past gradients to a fixed size rather than accumulating all past squared gradients [ 43 ]. The vector of a diagonal matrix (

with the decaying average over past squared gradients was represented as shown in Equation (8):

…”

Section: Methodsmentioning

confidence: 99%

Deep Learning-Based Cattle Vocal Classification Model and Real-Time Livestock Monitoring System with Noise Filtering

Park

Kim

Moon

et al. 2021

Animals

View full text Add to dashboard Cite

The priority placed on animal welfare in the meat industry is increasing the importance of understanding livestock behavior. In this study, we developed a web-based monitoring and recording system based on artificial intelligence analysis for the classification of cattle sounds. The deep learning classification model of the system is a convolutional neural network (CNN) model that takes voice information converted to Mel-frequency cepstral coefficients (MFCCs) as input. The CNN model first achieved an accuracy of 91.38% in recognizing cattle sounds. Further, short-time Fourier transform-based noise filtering was applied to remove background noise, improving the classification model accuracy to 94.18%. Categorized cattle voices were then classified into four classes, and a total of 897 classification records were acquired for the classification model development. A final accuracy of 81.96% was obtained for the model. Our proposed web-based platform that provides information obtained from a total of 12 sound sensors provides cattle vocalization monitoring in real time, enabling farm owners to determine the status of their cattle.

show abstract

with the decaying average over past squared gradients was represented as shown in Equation (8):

…”

Section: Methodsmentioning

confidence: 99%

Deep Learning-Based Cattle Vocal Classification Model and Real-Time Livestock Monitoring System with Noise Filtering

Park

Kim

Moon

et al. 2021

Animals

View full text Add to dashboard Cite

show abstract

“…Since the training sample data is large, the mini‐batch gradient descent (MBGD) algorithm is used as the optimization algorithm, which can reduce the calculation cost and increase the operation speed. Moreover, all the models are trained using the RMSprop optimizer …”

Section: Artificial Neural Network Approaches and Model For Short‐termentioning

confidence: 99%

An effective deep learning neural network model for short‐term load forecasting

Wang

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

Summary Energy load forecasting plays an important role in the smart grid, which can affect the promoting energy production and consumption decision‐making processes. In this paper, the state‐of‐the‐art deep learning (DL) neural models are used in the short‐term load forecasting, including the multilayer perceptron (MLP), the convolutional neural network (CNN), and the long short‐term memory (LSTM). A novel loss function is proposed for the load forecasting, and two commonly used benchmarks are used to verify the validity of the proposed function. The simulation results show that the mean absolute percentage error (MAPE) of the proposed loss function is 19.63% lower than cross‐entropy and 2.34% lower than mean absolute error (MAE). We compared the mentioned neural networks in different aspects, and the results show that in energy load forecasting, CNN has superior performance than MLP and LSTM in terms of high accuracy and robustness to weather changes.

show abstract

“…The training parameters of all the CNNs are the same. The learning rate is 0.01, total training steps are 50,000 and the optimizer is root mean square prop (RMSProp) [28]. The results are shown in Table 2.…”

Section: Event Recognitionmentioning

confidence: 99%

An Event Recognition Method for Φ-OTDR Sensing System Based on Deep Learning

Shi

Wang

Zhao

et al. 2019

Sensors

114

View full text Add to dashboard Cite

Phase-sensitive optical time domain reflectometer (Φ-OTDR) based distributed optical fiber sensing system has been widely used in many fields such as long range pipeline pre-warning, perimeter security and structure health monitoring. However, the lack of event recognition ability is always being the bottleneck of Φ-OTDR in field application. An event recognition method based on deep learning is proposed in this paper. This method directly uses the temporal-spatial data matrix from Φ-OTDR as the input of a convolutional neural network (CNN). Only a simple bandpass filtering and a gray scale transformation are needed as the pre-processing, which achieves real-time. Besides, an optimized network structure with small size, high training speed and high classification accuracy is built. Experiment results based on 5644 events samples show that this network can achieve 96.67% classification accuracy in recognition of 5 kinds of events and the retraining time is only 7 min for a new sensing setup.

show abstract

Comparison of the stochastic gradient descent based optimization techniques

Cited by 73 publications

References 1 publication

Deep Learning-Based Cattle Vocal Classification Model and Real-Time Livestock Monitoring System with Noise Filtering

Deep Learning-Based Cattle Vocal Classification Model and Real-Time Livestock Monitoring System with Noise Filtering

An effective deep learning neural network model for short‐term load forecasting

An Event Recognition Method for Φ-OTDR Sensing System Based on Deep Learning

Contact Info

Product

Resources

About