Multi-stage stochastic gradient method with momentum acceleration

Luo, Zhiwei; Chen, Siyu; Qian, Yuntao; Hou, Yueen

doi:10.1016/j.sigpro.2021.108201

Cited by 6 publications

(2 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To accelerate the SG algorithm, modifying step size and modifying gradient are two common methods. [18][19][20] Among these acceleration methods, the multi-innovation (MI) technology is extensively employed in recent years. [21][22][23] For example, a three-stage MI stochastic gradient algorithm was developed for time-series models, 21 an MI fractional-order adaptive algorithm was presented for nonlinear systems" 22 and a correlation analysis-based MI algorithm was proposed for linear errors-in-variables systems.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Time‐delay and parameter estimation for an ARX model based on copula theory and Armijo criterion and their applications in the modeling of the dynamics of the UAV

Jing

Chen

Liu

2022

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

Time‐delay dynamic systems are widely existed in industrial applications owing to the measure, control or other processes. To carry out system analysis, control and fault diagnosis, the identification of time‐delay systems becomes more and more important. This article considers the identification of time‐delay ARX models based on a novel two‐stage algorithm. Firstly, a 2‐copula criterion based time‐delay estimation method is presented by using a measure of dependence between the model input and output. This method can obtain the time delay without the estimates of the parameters. Secondly, a multi‐gradient algorithm with adaptive stacking length is studied. This algorithm accelerates traditional stochastic gradient algorithm by taking several recent gradients in each iteration. The stacking length, that is, the number of the gradient used in a step, is determined by the Armijo criterion. The proposed algorithm is validated by numerical experiments and the modeling of unmanned aerial vehicle data.

show abstract

Section: Introductionmentioning

confidence: 99%

“…But this algorithm converges slowly. To accelerate the SG algorithm, modifying step size and modifying gradient are two common methods 18‐20 . Among these acceleration methods, the multi‐innovation (MI) technology is extensively employed in recent years 21‐23 .…”

Section: Introductionmentioning

confidence: 99%