A new pruning algorithm for Feedforward Neural Networks

Ai, Fangju

doi:10.1109/iwaci.2011.6160018

Cited by 4 publications

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Atualmente, observa-se um grande interesse por algoritmos de aprendizagem que determinem tanto a estrutura como os pesos da rede neural. Por exemplo, algoritmos construtivos (crescimento, poda, crescimento-poda) e algoritmos evolucionários (Liu et al;Fangju;Miche et al;Feng et al;2009;2009a,b;Javan et al;Pisani e Lorena; Sistemas evolutivos são sistemas adaptativos de alto nível, pois eles determinam sua estrutura e respectivos parâmetros de forma simultânea, gradual e incremental. Portanto, são sistemas capazes de aprender a partir de um fluxo de dados, o que é muito conveniente em ambientes on-line ou tempo real.…”

Section: Os-elmunclassified

“…Nos últimos anos, diversos estudos investigaram a auto determinação da arquitetura de uma NN, dependente dos dados de entrada da rede neural ou do desempenho da rede em uma determinada aplicação. Os sistemas propostos baseiam-se em algoritmos construtivos ou de crescimento, poda, construtivo-poda e algoritmos evolucionários (Liu et al;Fangju;Miche et al;Feng et al;2009;2009a,b;Rivals e Personnaz;2002;Javan et al;Gao;2005 Esta seção propõe uma rede neural evolutiva de única camada intermediária com aprendizado extremo recursivo (eNNEL, evolving Neural Network with recursive Extreme Learning), que determina o número de neurônios na camada intermediária através de um algoritmo do tipo construtivo e construtivo-poda (Rosa, Gomide, Dovzan e Skrjanc;. Diferente dos algoritmos construtivos, de poda e evolucionários, a eNNEL determina o número de neurônios de nuvens no instante t. Um neurônio sigmoidal é aquele que tem função de ativação sigmoidal.…”

Section: Rede Neural Evolutiva Com Aprendizado Extremounclassified

See 1 more Smart Citation

Redes neurais evolutivas com aprendizado extremo recursivo

Rosa¹

View full text Add to dashboard Cite

This dissertation studies a class of evolving neural networks for system modeling from data streams. The class encompasses single hidden layer feedforward neural networks with variable and online definition of the number of hidden neurons.Evolving neural network learning uses clustering methods to estimate the number of hidden neurons simultaneously with extreme learning algorithms to compute the weights of the hidden and output layers. A particular case is when the evolving network keeps the number of hidden neurons fixed. In this case, the number of hidden neurons is found a priori, and the hidden and output layer weights updated as data are input. Clustering and extreme learning algorithms are recursive. Therefore, the learning process may occur online or real-time using data stream as input.Two evolving neural networks are suggested in this dissertation. The first is an evolving hybrid fuzzy neural network with unineurons in the hidden layer. Unineurons are fuzzy neurons whose synaptic processing is performed using uninorms. The output neurons are sigmoidals. A recursive clustering algorithm based on density and data clouds is used to granulate the input-output space, and to estimate the number of hidden neurons of the network. Each cloud corresponds to a hidden neuron. The weights of the hybrid fuzzy neural network are found using the extreme learning machine and the weighted recursive least squares algorithm. The second network is an evolving multilayer neural network with sigmoidal hidden and output neurons. Like the hybrid neural fuzzy network, clouds granulate the input-output space and gives the number of hidden neurons. The algorithm to compute the network weights is the same recursive version of the extreme learning machine. A variation of the adaptive OS-ELM (online sequential extreme learning machine) network is also suggested.Similarly as the original, the new OS-ELM fixes the number of hidden neurons, but uses sigmoidal instead of linear neurons in the output layer. The new OS-ELM also uses weighted recursive least square. ix The hybrid and neural networks were evaluated using two classic benchmarks: the gas furnace identification using the Box-Jenkins data, and forecasting of the chaotic Mackey-Glass time series. Synthetic data were produced to evaluate the neural networks when modeling systems with concept drift and concept shift. This a modeling circumstance in which system structure and parameters change simultaneously.Evaluation was done using the root mean square error and the Deibold-Mariano statistical test. The performance of the evolving and adaptive neural networks was compared against neural network with extreme learning, and evolving modeling methods representative of the current state of the art. The results show that the evolving neural networks and the adaptive network suggested in this dissertation are competitive and have similar or superior performance than the evolving approaches proposed in the literature.

show abstract