Comparative Analysis of Gaussian Process Regression Based Extreme Learning Machine

Abstract

Abstract:It is an effective way to overcome the randomization sensibility of extreme learning machine (ELM) by using Gaussian process regression (GPR) to optimize the output-layer weights. The key of GPR based ELM (GPRELM) is the selection of kernel function which is used to measure the similarity between different hidden-layer output vectors. In this paper, we conduct an experimental analysis to compare the classification performances of radial basis function (RBF) kernel and polynomial (Poly) kernel based GPRELMs. The comparative results on 24 UCI data sets reveal that: (1) GPRELMs have the serious over-fitting; (2) GPRELMs can get the better classification accuracies with less hidden-layer nodes in comparison with the original ELM; and (3) the smaller regularization factors usually bring about the higher training accuracies for GPRELMs, while the larger regularization factors usually result in the higher testing accuracies. All these conclusions provide the useful enlightenments and instructions for the theoretical studies and practical applications of GPRELMs.