New Approaches for Choosing the Ridge Parameters

Abstract

Consider the standard multiple linear regression model y = xβ +ε. If the correlation matrix x t x is ill-conditioned, the ordinary least squared estimate (ols)β of β is not the best choice. In this paper, multiple regularization parameters for different coefficients in ridge regression are proposed. The Mean Squared Error (MSE) of a ridge estimate based on the multiple regularization parameters is less than or equal to the MSE of the ridge estimate based on [2]. The proposed approach, depending on the condition numbers, leave's zero for the largest eigenvalue of x t x and gives the largest value for the smallest eigenvalue of x t x. Furthermore, if x t x is nearly a unit matrix, x t x is not an ill-conditioned one. The proposed approach gives approximately the same results as the ols estimates. The proposed approach can also be modified to give other new ridge parameters. The modified approach depends on the eigenvalues of x t x and differ from the ridge parameter proposed by [5] by a factor. The body fat data set has severe multicollinearity and is used to compare different approaches.Mathematical Subject Classification: Primary 62J02 Secondary 62J12

Biclustering Models Under Collinearity in Simulated Biological Experiments

“…There are several methods that can be used to ascertain the degree of collinearity in a dataset, such as Variance Inflation Factor, Condition Index, Condition Number, and Tolerance [36,37]. In this study, Condition Number (CN ) was used as the test statistic to gauge the level of collinearity.…”

Farklı regresyon modelleriyle kestirilen zenit troposferik gecikmelerin değerlendirilmesi

“…(hataların kareleri toplamı) (düzeltme terimi) Düzeltme terimi içerisinde yer alan λ2 parametresi β1, β2, …, βj katsayılarını sıfıra doğru daraltmakta olup düzeltme teriminin regresyon katsayılarını kontrol etmeyi sağlar. λ2 parametresi sadece β1, β2, …, βj katsayıları ile etkileşim halindedir β0'ı etkilememektedir [32]. Ridge regresyonu için düzeltme terimindeki λ2'nin 0'a eşit olması, parametrenin hiçbir etkisinin olmadığı ve eşitliğin EKK yöntemine dönüştüğü anlamına gelmektedir.…”

Forecasting Sales of Truck Components: A Machine Learning Approach

“…3) Ridge Regression: Ridge regression is a method used for the analysis of the multiple regression data that suffers from multi-collinearity [13]. When the occurrence of the multi-collinearity happens, least squares are unbiased but their variances are too large.…”

“…When the occurrence of the multi-collinearity happens, least squares are unbiased but their variances are too large. In this case, by using this method, the optimal prediction result may not be achievable [13]. By the addition of a degree of bias to the regression estimates, the standard error is reduced by the ridge regression.…”

“…By the addition of a degree of bias to the regression estimates, the standard error is reduced by the ridge regression. It is expected that the net effect will be to give estimates that are more reliable [13]. The cost function for ridge regression is as follows:…”