Design of Momentum Fractional Stochastic Gradient Descent for Recommender Systems

Khan, Zeshan Aslam; Zubair, Syed; Alquhayz, Hani; Azeem, Muhammad; Ditta, Allah

doi:10.1109/access.2019.2954859

Cited by 28 publications

(13 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, we use different optimizers (including Stochastic gradient descent method, SGD [34][35][36][37][38], momentum Stochastic gradient descent method [39][40][41][42], and adaptive moment estimation method, Adam [43][44]) to optimize the convolutional neural network constructed above. The SGD makes the loss function converge slowly, the convergence rate of the momentum SGD method in the early stage is similar to that of the Stochastic gradient descent method, but it can effectively converge the loss function in the later stage; The adaptive moment estimation method (Adam) can make the loss function converge quickly in the initial stage of neural network training, but it cannot converge further after the loss function converges to a certain extent in the later stage.…”

Section: Resultsmentioning

confidence: 99%

Metamaterials Design Method based on Deep learning Database

Zhou

Xiao

Wang

2022

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

In recent years, deep learning has risen to the forefront of many fields, overcoming challenges previously considered difficult to solve by traditional methods. In the field of metamaterials, there are significant challenges in the design and optimization of metamaterials, including the need for a large number of labeled data sets and one-to-many mapping when solving inverse problems. Here, we will use deep learning methods to build a metamaterial database to achieve rapid design and analysis methods of metamaterials. These technologies have significantly improved the feasibility of more complex metamaterial designs and provided new metamaterial design and analysis ideas.

show abstract

Section: Resultsmentioning

confidence: 99%

Metamaterials Design Method based on Deep learning Database

Zhou

Xiao

Wang

2022

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

show abstract

“…To improve the training ability of a neural network, this paper proposes a fractional-order gradient descent with the momentum method for training RBF neural networks. Compared with reference [12], the convergence of the proposed algorithm is proved in this paper. Compared with references [13,14], this paper uses both the momentum and gradient descent method.…”

Section: Introductionmentioning

confidence: 85%

Estimation of Navigation Mark Floating Based on Fractional-Order Gradient Descent with Momentum for RBF Neural Network

Fang

2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

To address the difficulty of estimating the drift of the navigation marks, a fractional-order gradient with the momentum RBF neural network (FOGDM-RBF) is designed. The convergence is proved, and it is used to estimate the drifting trajectory of the navigation marks with different geographical locations. First, the weight of the neural network is set. The navigation mark’s meteorological, hydrological, and initial position data are taken as the input of the neural network. The neural network is trained and used to estimate the mark’s position. The navigation mark’s position is taken at a later time as the output of the neural network. The difference between the later position and the estimated position obtained from the neural network is the error function of the neural network. The influence of sea conditions and months are analyzed. The experimental results and error analysis show that FOGDM-RBF is better than other algorithms at trajectory estimation and interpolation, has better accuracy and generalization, and does not easily fall into the local optimum. It is effective at accelerating convergence speed and improving the performance of a gradient descent method.

show abstract

“…The Adam algorithm [39] is different from traditional stochastic gradient descent [40], [41]. In stochastic gradient descent, a single learning rate is used to update all weights, and the learning rate does not change during the training process.…”

Section: Overall Optimization Of the Rbfnnmentioning

confidence: 99%

Radial Basis Function Neural Network Model for Dissolved Oxygen Concentration Prediction Based on an Enhanced Clustering Algorithm and Adam

Sun

Feng

2021

IEEE Access

View full text Add to dashboard Cite

In fishery aquaculture, water quality directly determines the economic benefits of aquatic products, and dissolved oxygen is an important factor affecting water quality. To accurately grasp the trends of variation in dissolved oxygen, a dissolved oxygen concentration forecasting model based on an enhanced clustering algorithm and Adam with a radial basis function neural network (ECA-Adam-RBFNN) is proposed. An enhanced clustering algorithm (ECA) combining K-means with ant colony optimization is introduced in place of random selection to determine the center positions of the neural network hidden layer units. If the number of center points is too high, the neural network will be overfit, whereas if it is too low, sudden changes will appear in the results. Once the hidden layer centers have been determined, the radial basis function (RBF) width is calculated from the maximum center distance and the number of center points to avoid the two extreme cases of RBF that are too peaked or flat. The recursive least squares (RLS) algorithm is introduced to obtain the connection weights from the hidden layer to the output layer. The Adam algorithm is introduced to iteratively differentiate the objective function to adjust the center values, weights and width while adaptively varying the learning rates for these three types of parameters. Finally, the improved forecasting algorithm is applied for the prediction of the dissolved oxygen concentration in fishery aquaculture. The experimental results show that under identical conditions, compared with a long short-term memory (LSTM) network, a backpropagation neural network (BPNN), a traditional RBF neural network, a support vector regression (SVR) model, an autoregressive integrated moving average (ARIMA), K-MLPNN (K-means muhilayer perceptron neural networks), and SC-K-means-RBF model, the improved algorithm achieves significant reductions in the mean absolute error (MAE), mean absolute percentage error (MAPE) and root mean square error (RMSE) as model evaluation indicators.

show abstract

Design of Momentum Fractional Stochastic Gradient Descent for Recommender Systems

Cited by 28 publications

References 57 publications

Metamaterials Design Method based on Deep learning Database

Metamaterials Design Method based on Deep learning Database

Estimation of Navigation Mark Floating Based on Fractional-Order Gradient Descent with Momentum for RBF Neural Network

Radial Basis Function Neural Network Model for Dissolved Oxygen Concentration Prediction Based on an Enhanced Clustering Algorithm and Adam

Contact Info

Product

Resources

About