A Homotopy Method for Parameter Estimation of Nonlinear Differential Equations with Multiple Optima

Hao, Wenrui

doi:10.1007/s10915-017-0518-4

Cited by 8 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Parameter estimation often requires a tremendous number of model evaluations to obtain the solution information on the parameter space [21,36,37]. However, this large number of model evaluations becomes very difficult and even impossible for large-scale computational models [38,39].…”

Section: (B) Parameter Estimationmentioning

confidence: 99%

“…In order to solve these challenges, in this paper, we will present a new training algorithm based on the homotopy continuation method [13][14][15], which has been successfully used to study nonlinear problems such as nonlinear differential equations [16][17][18], hyperbolic conservation laws [19,20], data-driven optimization [21,22], physical systems [23] and some more complex free boundary problems arising from biology [24,25]. In order to tackle the nonlinear optimization problem in DNN, the homotopy training algorithm (HTA) is designed and shows efficiency and feasibility for fully connected neural networks with complex structures.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A homotopy training algorithm for fully connected neural networks

Chen

Hao

2019

Proc. R. Soc. A.

Self Cite

View full text Add to dashboard Cite

In this paper, we present a Homotopy Training Algorithm (HTA) to solve optimization problems arising from fully connected neural networks with complicated structures. The HTA dynamically builds the neural network starting from a simplified version and ending with the fully connected network via adding layers and nodes adaptively. Therefore, the corresponding optimization problem is easy to solve at the beginning and connects to the original model via a continuous path guided by the HTA, which provides a high probability of obtaining a global minimum. By gradually increasing the complexity of the model along the continuous path, the HTA provides a rather good solution to the original loss function. This is confirmed by various numerical results including VGG models on CIFAR-10. For example, on the VGG13 model with batch normalization, HTA reduces the error rate by 11.86% on test dataset compared with the traditional method. Moreover, the HTA also allows us to find the optimal structure for a fully connected neural network by building the neutral network adaptively.

show abstract