Estimation of Deep Neural Networks Capabilities Using Polynomial Approach

Różycki, Paweł; Kolbusz, Janusz; Korostenskyi, Roman; Wilamowski, Bogdan M.

doi:10.1007/978-3-319-39378-0_13

Cited by 3 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sigmoid activation functions in neural networks have been expanded into Taylor series for several purposes, such as concentration estimation of toluene gas from the trend of the transient sensor responses , hardware implementation of neural network models , and learning of deep neural networks . We reported preliminary studies for deriving a NARX model from a neural network using Taylor series expansion of sigmoid activation functions .…”

Section: Introductionmentioning

confidence: 99%

“…It is to be noted that NARX/NARMAX models allow the analysis of nonlinear systems in the frequency domain. Generalized frequency response functions (GFRFs) [39,40] and nonlinear output frequency response functions (NOFRFs) [41] can be easily be computed directly from concentration estimation of toluene gas from the trend of the transient sensor responses [58], hardware implementation of neural network models [59][60][61], and learning of deep neural networks [62,63]. We reported preliminary studies for deriving a NARX model from a neural network using Taylor series expansion of sigmoid activation functions [64][65][66].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Derivation of NARX models by expanding activation functions in neural networks

Inaoka

Kobayashi

Nebuya

et al. 2019

IEEJ Transactions Elec Engng

View full text Add to dashboard Cite

A method was developed to derive an NARX model from a neural network so that the usability of open‐source libraries for network learning was combined with the NARX advantage of revealing the system structure. After the neural network model was trained on input and output data, the sigmoid activation functions were expanded into Taylor series. Candidate parameters in the NARX model were calculated from the connection weights in the neural network and coefficients in the series. The NARX model structure was determined by the extended least‐squares (ELS) method and the Bayesian information criterion (BIC). Correct NARX models were successfully detected in computer simulations and an experiment. The developed method can be used for any activation functions, including the sigmoid function, if they are expanded into Taylor series. © 2019 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

show abstract

Section: Introductionmentioning

confidence: 99%