Binary/ternary extreme learning machines

Abstract

a b s t r a c tIn this paper, a new hidden layer construction method for Extreme Learning Machines (ELMs) is investigated, aimed at generating a diverse set of weights. The paper proposes two new ELM variants: Binary ELM, with a weight initialization scheme based on f0; 1g-weights; and Ternary ELM, with a weight initialization scheme based on fÀ1; 0; 1g-weights. The motivation behind this approach is that these features will be from very different subspaces and therefore each neuron extracts more diverse information from the inputs than neurons with completely random features traditionally used in ELM. Therefore, ideally it should lead to better ELMs. Experiments show that indeed ELMs with ternary weights generally achieve lower test error. Furthermore, the experiments show that the Binary and Ternary ELMs are more robust to irrelevant and noisy variables and are in fact performing implicit variable selection. Finally, since only the weight generation scheme is adapted, the computational time of the ELM is unaffected, and the improved accuracy, added robustness and the implicit variable selection of Binary ELM and Ternary ELM come for free.

Air quality forecasting using neural networks

“…By adjusting the value of γ, the regularized version of ELM can prevent itself from overfitting [39,57]. Furthermore, adding a linear component in the ELM might be helpful in some cases, making the model as:…”

Online sequential prediction of bearings imbalanced fault diagnosis by extreme learning machine

“…Online sequential extreme learning machine (OS-ELM) [13], a sequential modification of ELM proposed by Huang et al [14], can deal the sequential arriving data with good generalization and fast learning speed. It has widespread use in many applications, especially in the field of fault diagnosis [15][16][17]. However, although OS-ELM works effectively on online sequential data, it tends to get poorer classification effect, especially the minority classified accuracy [18], when applied in the severely imbalanced data.…”

Comparative Analysis of Gaussian Process Regression Based Extreme Learning Machine

“…Ref. [5] proposed two weight initialization schemes, i.e., binary ELM based on {0, 1}-weights and ternary ELM based on {-1, 0, 1}-weights, to improve the diversity of neurons in the hidden layer. For binary/ternary ELMs, the necessary optimizations are also required to select the better…”