Unbiased Estimation for Linear Regression When n < v

Abstract

In this paper a new method is proposed for solving the linear regression problem when the number of observations n is smaller than the number of predictors v. This method uses the idea of graphical models and provides unbiased parameter estimates under certain conditions, while existing methods such as ridge regression, least absolute shrinkage and selection operator (LASSO) and least angle regression (LARS) give biased estimates. Also the new method can provide a detailed graphical correlation structure for the predictors, therefore the real causal relationship between predictors and response could be identified. In contrast, existing methods often cannot identify the real important predictors which have possible causal effects on the response variable. Unlike the existing methods based on graphical models, the proposed method can identify the potential networks while doing regression even if the data do not follow a multivariate distribution. The new method is compared with some existing methods such as ridge regression, LASSO and LARS by using simulated and real data sets. Our experiments reveal that the new method outperforms all the other methods when n < v.

Hybrid graphical least square estimation and its application in portfolio selection

“…The proof for a very special case of the GLSE method (the graph with only two cliques) has been shown in Aldahmani and Dai (2015). Here we provide a more general proof with respect to Theorem 2.1, applicable to GLSE with more than two cliques.…”

“…This study considers using an unbiased estimation method via graphical models for solving the linear regression problem when N < v. This estimation is named as Graphical Least Squares estimation (GLSE) (Aldahmani and Dai, 2015). A potential weakness of the GLSE is that, for a very large number of covariates/assets, the involved computation cost will be very heavy.…”