Universality of gradient descent neural network training

Intelligent classification of coal structure using multinomial logistic regression, random forest and fully connected neural network with multisource geophysical logging data

“…Neurons are combined according to a certain layered structure, and a neural network is thus created. In practice, as an activation function is required to satisfy the firing from neurons, the traditional sigmoid gradient function is prone to gradient disappearance (Welper, 2022), which makes it difficult to optimize important gradients.…”

“…For the intrinsic calculation of neural networks, back propagation is required to calculate the gradient correlation. However, due to the dependence of back propagation on memory and the consideration of overfitting, a direct error report is avoided (Welper, 2022), and an intermediate calculation is automatically released at the end of a gradient. The classification results for the coal structure can be obtained by inputting logging data through this training model.…”

A Sigma-Pi-Sigma Neural Network Model with Graph Regularity Term

“…Generally, we use gradient descent method to train the network [5]. However, when we use the squared error sum function to train the SPSNN, the inappropriate number of hidden layer neurons and the magnitude of the weights may lead to underfitting or overfitting of the model, which affects the performance and generalization ability of the model.…”

Short-Term Wind Power Prediction by an Extreme Learning Machine Based on an Improved Hunter–Prey Optimization Algorithm

“…The ELM algorithm was developed to address these issues. Since the 2 of 14 desired accuracy and stability of ELM cannot be met, and the weights and thresholds are randomly generated before training, it is typically optimized [8]. The benefits of the swarm intelligence optimization algorithm include strong parallelism, independent exploration, ease of use, quick convergence, etc.…”