Robust training of microwave neural network models using combined global/local optimization techniques

Ninomiya, Hiroshi; Wan, Shan Shan; Kabir, Humayun; Zhang, Xin; Zhang, Q.J.

doi:10.1109/mwsym.2008.4633002

Cited by 8 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This section discusses the performance of MoQ on five benchmark problems, i.e., three function approximation and two classification problems. For the function approximation problems, we use a benchmark problem with high nonlinearity [14] and two real-world problems such as the microwave circuit modeling [5,24]. For classification problems, 8 × 8 MNIST handwritten digits [25][26][27] and three-spirals [28]…”

Section: Simulation Resultsmentioning

confidence: 99%

“…This section briefly shows the QN based training algorithm and its accelerated methods. These methods have been commonly used as the training algorithms for problems with highly nonlinear properties [4,5,14].…”

Section: Related Workmentioning

confidence: 99%

“…Neural networks (NNs) have shown to be effective in numerous real-world applications [1,2]. For example, NNs are efficiently used in solving the function approximation problems such as microwave circuit modeling [3][4][5] which are problems with high nonlinear input-output characteristics or classification problems such as image classification [6] and the natural language processing [7]. Most of these applications require large NN models with massive amounts of training data to achieve good accuracies and low errors in minimum time.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Momentum acceleration of quasi-Newton based optimization technique for neural network training

Mahboubi

Indrapriyadarsini

Ninomiya

et al. 2021

NOLTA

View full text Add to dashboard Cite

This paper describes a momentum acceleration technique for quasi-Newton (QN) based neural network training and verifies its performance and computational complexity. Recently, Nesterov's accelerated quasi-Newton method (NAQ) has been introduced and shown that the momentum term is effective in reducing the number of iterations and the total training time by incorporating Nesterov's accelerated gradient into QN. However, the gradients had to be calculated two times in one iteration in the NAQ training. This increased the computation time of a training loop compared with the conventional QN. The proposed technique is an improvement to NAQ done by approximating the Nesterov's accelerated gradient as a linear combination of the current and previous gradients. As a result, the gradient is calculated only once per iteration similar to that of QN. The performance of the proposed algorithm is evaluated in comparison to conventional algorithms in neural networks training on two types of problems -function approximation problems with high nonlinearity and classification problems. The results show a significant acceleration in the computation time without losing the quality of the solution compared with conventional training algorithms.

show abstract

Section: Simulation Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Momentum acceleration of quasi-Newton based optimization technique for neural network training

Mahboubi

Indrapriyadarsini

Ninomiya

et al. 2021

NOLTA

View full text Add to dashboard Cite

show abstract

Distributed robust training of multilayer neural netwroks using normalized risk-averting error

Ninomiya

2014

2014 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB)

View full text Add to dashboard Cite

A hybrid global/local optimization technique for robust training of microwave neural network models

Ninomiya

2009

2009 IEEE Congress on Evolutionary Computation

View full text Add to dashboard Cite

This paper describes a new technique for training microwave neural network models. The proposed technique combines quasi-Newton algorithm with a global optimization algorithm called Particle Swarm Optimization (PSO). The quasi-Newton process for searching optimal solutions is incorporated into PSO to speed up local search, while the PSO performs global search avoid being trapped in local minima of training. The overall algorithm iterates between quasi-Newton and PSO. Neural network training for microwave circuit modeling, such as waveguide and microstrip examples is presented, demonstrating that the proposed algorithm achieves more accurate models than the conventional gradient based technique and the conventional PSOs.

show abstract

Robust training of microwave neural network models using combined global/local optimization techniques

Cited by 8 publications

References 6 publications

Momentum acceleration of quasi-Newton based optimization technique for neural network training

Momentum acceleration of quasi-Newton based optimization technique for neural network training

Distributed robust training of multilayer neural netwroks using normalized risk-averting error

A hybrid global/local optimization technique for robust training of microwave neural network models

Contact Info

Product

Resources

About