Extreme learning machine versus classical feedforward network

Abstract

Our research is devoted to answering whether randomisation-based learning can be fully competitive with the classical feedforward neural networks trained using backpropagation algorithm for classification and regression tasks. We chose extreme learning as an example of randomisation-based networks. The models were evaluated in reference to training time and achieved efficiency. We conducted an extensive comparison of these two methods for various tasks in two scenarios: $$\bullet$$ ∙ using comparable network capacity and $$\bullet$$ ∙ using network architectures tuned for each model. The comparison was conducted on multiple datasets from public repositories and some artificial datasets created for this research. Overall, the experiments covered more than 50 datasets. Suitable statistical tests supported the results. They confirm that for relatively small datasets, extreme learning machines (ELM) are better than networks trained by the backpropagation algorithm. But for demanding image datasets, like ImageNet, ELM is not competitive to modern networks trained by backpropagation; therefore, in order to properly address current practical needs in pattern recognition entirely, ELM needs further development. Based on our experience, we postulate to develop smart algorithms for the inverse matrix calculation, so that determining weights for challenging datasets becomes feasible and memory efficient. There is a need to create specific mechanisms to avoid keeping the whole dataset in memory to compute weights. These are the most problematic elements in ELM processing, establishing the main obstacle in the widespread ELM application.

Spectral library transfer between distinct laser-induced breakdown spectroscopy systems trained on simultaneous measurements

“…The utilized model is an extreme learning machine (architecture and forward pass are analogical to the MLP but weights are not trainable), which was shown to be signicantly outperformed by MLPs (trained by the stochastic gradient descent) for large datasets. 24 In LIBS, CT was studied in ref. 19, as an example of a general manifold alignment problem.…”