Hessian semi-supervised extreme learning machine

Krishnasamy, Ganesh; Raveendran, P.

doi:10.1016/j.neucom.2016.05.039

Cited by 18 publications

(8 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It should be noted that the neural network trained by the Hessian (H) in [17][18][19][20] and using (15), (18), (14), (9), (10) had l = 6 neurons in its hidden layer, a tuning factor of α = 0.0004, and a number of epochs of e = 40.…”

Section: Results Of the Comparisonmentioning

confidence: 99%

“…In this section, we compare steepest descent (SD), steepest descent with mini-batches (SDMB) from [9][10][11][12], the Hessian (H) from [17][18][19][20], and the Hessian with mini-batches (HMB) from this investigation for electrical demand prediction. The goal of these algorithms is that the neural network output q l must reach the target t l as soon as possible.…”

Section: Comparisonsmentioning

confidence: 99%

“…There have been several applications of the Hessian. In [17][18][19][20], the Hessian was used for tuning. In [21][22][23][24], the Hessian was employed for segmentation.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Hessian with Mini-Batches for Electrical Demand Prediction

et al. 2020

View full text Add to dashboard Cite

The steepest descent method is frequently used for neural network tuning. Mini-batches are commonly used to get better tuning of the steepest descent in the neural network. Nevertheless, steepest descent with mini-batches could be delayed in reaching a minimum. The Hessian could be quicker than the steepest descent in reaching a minimum, and it is easier to achieve this goal by using the Hessian with mini-batches. In this article, the Hessian is combined with mini-batches for neural network tuning. The discussed algorithm is applied for electrical demand prediction.

show abstract

Section: Results Of the Comparisonmentioning

confidence: 99%

Section: Comparisonsmentioning

confidence: 99%

See 1 more Smart Citation

Hessian with Mini-Batches for Electrical Demand Prediction

et al. 2020

View full text Add to dashboard Cite

show abstract

“…However, Hessian regularization is not robust for many semisupervised tasks. Especially, due to the result of sampling datasets is usually not dense which lead to inaccurate estimation [26]. In this section, we present the LHRSS-ELM algorithm in detail, which integrates the Laplacian and Hessian regularization term that takes advantage of manifold structure of the data space to improve the performance of traditional ELM.…”

Section: B Lhrss-elm Formulationmentioning

confidence: 99%

“…Laplacian regularization is one of the most popular manifold regularization, utilizing graph Laplacian to determine the geometry of the underlying manifold, has been successful used in semi-supervised tasks [19], [34], [35], [37], [38]. However, if only a few of labeled data available that the performance will be worsen due to lacking of extrapolating power, biased the solution towards a constant function and cannot preserve the local topology architecture [26], [25]. Another regularization manifold called Hessian regularization can make the learned functions whose values vary linearly along the data manifold, while the Hessian operator is time-consuming and not have efficient results, and it is not robust and feasible in computation cost.…”

Section: Introductionmentioning

confidence: 99%

A Hybrid Regularization Semi-Supervised Extreme Learning Machine Method and Its Application

et al. 2019

View full text Add to dashboard Cite

Semi-supervised extreme learning machine (SS-ELM) has been applied to many classification and regression assignments with high performance, in which both the labeled and unlabeled data are exploited to enhance accuracy and computation efficiency. The Laplacian manifold regularization method has been incorporated to explore the geometry of the underlying manifold structure. However, the Laplacian manifold regularization lacks the extrapolating ability and biases the solution to a real constant function. In this paper, we propose a novel algorithm, the Laplacian-Hessian regularization SS-ELM (LHRSS-ELM), to enhance the performance of conventional SS-ELM. The main advantages of LHRSS-ELM are as follows: 1) LHRSS-ELM exhibits the learning capability and computational efficiency of traditional SS-ELMs; 2) LHRSS-ELM algorithm combines both Laplacian and Hessian term to enhance the extrapolating power, accuracy, and robustness and also show significant performance in multiclass classification tasks; and 3) for the purpose of pursuing the best pair of hyperparameters to establish a comparable model, we dynamically update them from sequences. The proposed algorithm is evaluated on publicly available data sets and further applied for the state classification of superheating degree in the aluminum electrolysis process. The experimental results demonstrate that the proposed mechanism is superior to the existing state-of-the-art semi-supervised learning algorithms in the matter of accuracy and robustness.

show abstract