Singular Value Decomposition update and its application to (Inc)-OP-ELM

Grigorievskiy, Alexander; Miché, Yoan; Käpylä, Maarit J.; Lendasse, Amaury

doi:10.1016/j.neucom.2015.03.107

Cited by 11 publications

(6 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the SVD update by (pp. 101-102, [19]) was readily available for Python 2.7 (link in source for original) and had to be modified to use Python 3.…”

Section: Details On the Methodsmentioning

confidence: 99%

“…One possibility to reduce the computational burden in the linear distance regression is to use the iterative Singular Value Decomposition (SVD) update (pp. 101-102, [19]), which tries to reduce the total computational complexity by using the previously computed regression model. A more straightforward approach to speed up the computation is to use randomized SVD (pp.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Do Randomized Algorithms Improve the Efficiency of Minimal Learning Machine?

Linja

Hämäläinen

Nieminen

et al. 2020

MAKE

View full text Add to dashboard Cite

Minimal Learning Machine (MLM) is a recently popularized supervised learning method, which is composed of distance-regression and multilateration steps. The computational complexity of MLM is dominated by the solution of an ordinary least-squares problem. Several different solvers can be applied to the resulting linear problem. In this paper, a thorough comparison of possible and recently proposed, especially randomized, algorithms is carried out for this problem with a representative set of regression datasets. In addition, we compare MLM with shallow and deep feedforward neural network models and study the effects of the number of observations and the number of features with a special dataset. To our knowledge, this is the first time that both scalability and accuracy of such a distance-regression model are being compared to this extent. We expect our results to be useful on shedding light on the capabilities of MLM and in assessing what solution algorithms can improve the efficiency of MLM. We conclude that (i) randomized solvers are an attractive option when the computing time or resources are limited and (ii) MLM can be used as an out-of-the-box tool especially for high-dimensional problems.

show abstract

“…Moreover, the SVD update by (pp. 101-102, [19]) was readily available for Python 2.7 (link in source for original) and had to be modified to use Python 3.…”

Section: Details On the Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Do Randomized Algorithms Improve the Efficiency of Minimal Learning Machine?

Linja

Hämäläinen

Nieminen

et al. 2020

MAKE

View full text Add to dashboard Cite

show abstract

“…(5). The matrix + is the generalized Moore-Penrose inverse of the matrix , which can be found using the singular value decomposition method [20]. The data from a healthy wind turbine is selected as the training dataset to obtain the value of ̂ for the ELM model.…”

Section: A Data-driven Methodsmentioning

confidence: 99%

Data-Driven Condition Monitoring Approaches to Improving Power Output of Wind Turbines

Qian

Zhang

et al. 2019

IEEE Trans. Ind. Electron.

View full text Add to dashboard Cite

This paper presents data-driven approaches to improving active power output of wind turbines based on estimating their health condition. The main procedure includes estimations of fault degree and health condition level, and optimal power dispatch control. The proposed method can adjust active power output of individual turbines according to their health condition and can thus optimize the total energy output of wind farm. In the paper, extreme learning machine (ELM) algorithm and bonferroni interval are applied to estimate fault degree while analytic hierarchy process (AHP) is used to estimate the health condition level. A scheme for power dispatch control is formulated based on the estimated health condition. Models have been identified from supervisory control and data acquisition (SCADA) data acquired from an operational wind farm, which contains temperature data of gearbox bearing and generator winding. The results show that the proposed method can maximize the operation efficiency of the wind farm while significantly reduce the fatigue loading on the faulty wind turbines.

show abstract

“…Different approaches for calculating the proper regularization parameter ( λ) have been employed, such as the discrepancy principle, generalized cross validation (GCV), and the L-curve [24]. Appropriate model selection and parameter optimization can be achieved with leave-one-out cross validation (LOOCV), which consumes a large amount of computations [6,10,25,26], an adopted efficient computing method depending on the prediction sum of squares (PRESS) formula to calculate the cross validation minimum square error ( MSE CV ) utilizing Eq. ( 7):…”

Section: Tikhonov Regularization With L-curvementioning

confidence: 99%

Optimum, projected, and regularized extreme learning machine methods with singular value decomposition and L 2 -Tikhonov regularization

Shehab¹,

Kahraman²

2018

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

The theory and implementation of an extreme learning machine (ELM) have proved that it is a simple, efficient, and accurate machine learning methodology. In an ELM, the hidden nodes are randomly initiated and fixed without iterative tuning. However, the optimal hidden layer neuron number (Lopt) is the key to ELM generalization performance where initializing this number by trial and error is not reasonably satisfied. Optimizing the hidden layer size using the leave-one-out cross validation method is a costly approach. In this paper, a fast and reliable statistical approach called optimum ELM (OELM) was developed to determine the minimum hidden layer size that yields an optimum performance. Another improvement that exploits the advantages of orthogonal projections with singular value decomposition was proposed in order to tackle the problem of randomness and correlated features in the input data. This approach, named projected ELM (PELM), achieves more than 2% advance in average accuracy. The final contribution of this paper was implementing Tikhonov regularization in the form of the L 2-penalty with ELM (TRELM), which regularizes and improves the matrix computations utilizing the L-curve criterion and SVD. The L-curve, unlike iterative methods, can estimate the optimum regularization parameter by illustrating a curve with few points that represents the tradeoff between minimizing the training error and the residual of output weight. The proposed TRELM was tested in 3 different scenarios of data sizes: small, moderate, and big datasets. Due to the simplicity, robustness, and less time consumption of OELM and PELM, it is recommended to use them with small and even moderate amounts of data. TRELM demonstrated that when enhancing the ELM performance it is necessary to enlarge the size of hidden nodes (L). As a result, in big data, increasing L in TRELM is necessary, which concurrently leads to a better accuracy. Various well-known datasets and state-of-the-art learning approaches were compared with the proposed approaches.

show abstract

Singular Value Decomposition update and its application to (Inc)-OP-ELM

Cited by 11 publications

References 18 publications

Do Randomized Algorithms Improve the Efficiency of Minimal Learning Machine?

Do Randomized Algorithms Improve the Efficiency of Minimal Learning Machine?

Data-Driven Condition Monitoring Approaches to Improving Power Output of Wind Turbines

Optimum, projected, and regularized extreme learning machine methods with singular value decomposition and L 2 -Tikhonov regularization

Contact Info

Product

Resources

About