Online neuro-fuzzy CANFIS hidden-node teaching

Mizutani, Eiji; Fan, Jing-Yun Carey

doi:10.1109/fuzzy.2011.6007587

Cited by 1 publication

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, the terminal weights θ 2,3 in such an MLP are common to all data. Consequently, a TSK with two linear rules can solve the well-known XOR problem, a linearly non-separable two-class pattern classification problem, even with linear (hidden-node) functions at Stage 2 (see [13]). By contrast, any single-hidden-layer MLP with any number of linear hidden nodes (at Stage 2) is unable to solve XOR.…”

Section: B An Overlap Singularity In Adaptive-network Learningmentioning

confidence: 99%

A block-approximate local Hessian-matrix analysis for CANFIS neuro-fuzzy modular network learning

Mizutani

2012

2012 IEEE International Conference on Fuzzy Systems

Self Cite

View full text Add to dashboard Cite

We describe how to alleviate the problem of plateau phenomena that may arise in learning with a CANFIS neuro-fuzzy modular network. The network model consists of multiple "local-expert" MLPs (multilayer perceptrons) mediated by fuzzy membership functions. Even with such a complex modular architecture, our recently-developed secondorder stagewise backpropagation procedure efficiently evaluates the Hessian matrix of a given objective function, for which we employ the sum-squared-error measure. For concreteness, we use a small curve-fitting problem that allows us to demonstrate detailed analysis based on the Hessian matrix. In particular, we describe how to use a block-diagonal approximate local Hessian matrix associated with a bottleneck local-expert MLP that does not perform very well for a designated task. Since the "bad" performance implies relatively large residuals (or errors), the local Hessian matrix tends to be indefinite; therefore, it is worth exploiting the negative curvature to escape from the plateaus.

show abstract

Section: B An Overlap Singularity In Adaptive-network Learningmentioning

confidence: 99%