Two penalized mixed–integer nonlinear programming approaches to tackle multicollinearity and outliers effects in linear regression models

Abstract

In classical regression analysis, the ordinary least-squares estimation is the best strategy when the essential assumptions such as normality and independency to the error terms as well as ignorable multicollinearity in the covariates are met. However, if one of these assumptions is violated, then the results may be misleading. Especially, outliers violate the assumption of normally distributed residuals in the least-squares regression. In this situation, robust estimators are widely used because of their lack of sensitivity to outlying data points. Multicollinearity is another common problem in multiple regression models with inappropriate effects on the least-squares estimators. So, it is of great importance to use the estimation methods provided to tackle the mentioned problems. As known, robust regressions are among the popular methods for analyzing the data that are contaminated with outliers. In this guideline, here we suggest two mixed-integer nonlinear optimization models which their solutions can be considered as appropriate estimators when the outliers and multicollinearity simultaneously appear in the data set. Capable to be effectively solved by metaheuristic algorithms, the models are designed based on penalization schemes with the ability of down-weighting or ignoring unusual data and multicollinearity effects. We establish that our models are computationally advantageous in the perspective of the flop count. We also deal with a robust ridge methodology. Finally, three real data sets are analyzed to examine performance of the proposed methods.

Effect of Increasing Sample Size on Multi-Collinearity in Multilevel Non-Linear Model

“…For (Z'.Z) -1 to be invertible, it is essential that the columns of (Z'×Z) -1 exhibit linear independence. Consequently, the issue of multi-collinearity arises in the iterative optimisation processes of non-linear models [22].…”

Descent Spectral Versions of the Traditional Conjugate Gradient Algorithms With Application to Nonnegative Matrix Factorization

“…where is the penalty parameter and stands for the spectral condition number [26,27]. for an arbitrary positive definite matrix .…”

Potential Application of Alternative Materials for Organic Pollutant Removal

“…(9) q t = q avr * (1 − Exp(−k avr * t) n avr (10) q t = k wm * t 0.5 + B (RMSE), and error sum of squares (SSE) were used to evaluate the fit quality of the models, according to Eqs. ( 11)-( 14) (Ceylan, 2020;Doiron, 2019;Roozbeh et al, 2020).…”