Building a Robust Linear Model with Backward Elimination Procedure

Abstract

For building a linear prediction model, Backward Elimination (BE) is a computationally suitable stepwise procedure for sequencing the candidate predictors. This method yields poor results when data contain outliers and other contaminations. Robust model selection procedures, on the other hand, are not computationally efficient or scalable to large dimensions, because they require the fitting of a large number of submodels. Robust version of BE is proposed in this study, which is computationally very suitable and scalable to large high-dimensional data sets. Since BE can be expressed in terms of sample correlations, simple robustifications are obtained by replacing these correlations by their robust counterparts. A pairwise approach is used to construct the robust correlation matrix − not only because of its computational advantages over the d -dimensional approach, but also because the pairwise approach is more consistent with the idea of step-by-step algorithms. The performance of the proposed robust method is much better than standard BE.

Robust Multiple Linear Backward EliminationRegression

“…BE algorithm has been described with regard to sample means, standard deviations, and correlations 11 . It is well known that the sample mean and sample standard deviation are affected by outliers or other contaminations.…”

“…Our strategy is to use the BE model to sequence the candidate predictors to form a list, with the best predictors at the top. BE has been described using sample correlations and proposed a robust version of BE (RBE) that is based on two approaches to robust bivariate correlation estimates: adjusted winsorized correlation and Spearman rank correlation 11,12,13 . These two types of correlations are robust only to bivariate outliers.…”