Common Subset Selection of Inputs in Multiresponse Regression

Abstract

Abstract-We propose the Multiresponse Sparse Regression algorithm, an input selection method for the purpose of estimating several response variables. It is a forward selection procedure for linearly parameterized models, which updates with carefully chosen step lengths. The step length rule extends the correlation criterion of the Least Angle Regression algorithm for many responses. We present a general concept and explicit formulas for three different variants of the algorithm. Based on experiments with simulated data, the proposed method competes favorably with other methods when many correlated inputs are available for model construction. We also study the performance with several real data sets.

Forecasting the Yield Curve in a Data‐Rich Environment Using the Factor‐Augmented Nelson–Siegel Model

“…As R is multivariate in our setup, the most obvious application of the LARS algorithm is to feed the columns of R into the algorithm one at a time. An alternative approach is ‘multiresponse sparse regression’ (MRSR), proposed by Similä and Tikka () as an extension of LARS that allows for a multivariate response variable.…”

“…Denote the fitted value of R , based on the first m regressors chosen, by (with ), and denote the j th column of X by x ( j ) . Following Similä and Tikka (), the role of correlations in the description of the univariate case above is now played by , where || v || represents the L 2 vector norm . No other changes to the procedure are needed.…”

“…No other changes to the procedure are needed. An efficient algorithm to find the order in which variables are added is presented in Similä and Tikka ().…”

“…Here we examine this issue in more detail by comparing a number of data‐based variable selection methods with several approaches to construct factors from a large panel of variables. The methods for selecting variables in a data‐driven way are based on least angle regression (LARS) or multiresponse sparse regression (MRSR), as proposed by Efron et al () and Similä and Tikka (), respectively. The factor construction methods include principal component analysis (PCA), next to a number of alternative approaches discussed below.…”

A Survey ofL1Regression

“…Finally, the LARS/lasso approach can also be extended for multivariate (multiresponse) regression. Similä and Tikka () propose an extension of the LARS algorithm by modifying the correlation criterion between the predictors and the current residual (which depends on multiple outputs). Unfortunately, the exact regularization path can be recovered only when X is orthonormal.…”