Common Subset Selection of Inputs in Multiresponse Regression

Abstract

Abstract-We propose the Multiresponse Sparse Regression algorithm, an input selection method for the purpose of estimating several response variables. It is a forward selection procedure for linearly parameterized models, which updates with carefully chosen step lengths. The step length rule extends the correlation criterion of the Least Angle Regression algorithm for many responses. We present a general concept and explicit formulas for three different variants of the algorithm. Based on experiments with simulated data, the proposed method competes favorably with other methods when many correlated inputs are available for model construction. We also study the performance with several real data sets.

Forecasting the Yield Curve in a Data-Rich Environment Using the Factor-Augmented Nelson-Siegel Model

“…An alternative approach is "multiresponse sparse regression" (MRSR), proposed by Similä and Tikka (2006) as an extension of LARS that allows for a multivariate response variable.…”

“…Denote the fitted value of R, based on the first m regressors chosen, byR m (withR 0 = 0), and denote the j-th column of X by x (j) . Following Similä and Tikka (2006), the role of correlations in the description of the univariate case above is now played by…”

“…An efficient algorithm to find the order in which variables are added is presented in Similä and Tikka (2006).…”

“…The methods for selecting specific macroeconomic variables in a data-driven way are based on least angle regression (LARS) or multiresponse sparse regression (MRSR), as proposed by Efron et al (2004) and Similä and Tikka (2006), respectively. The factor construction methods include Principal Component Analysis (PCA), next to a number of alternative approaches discussed below.…”

Input selection and shrinkage in multiresponse linear regression

“…However, the greedy stepwise methods may fail to recognize important combinations of inputs, especially when the inputs are highly correlated (Derksen and Keselman, 1992). Better results can be obtained by incorporating shrinking in the selection strategy (Breiman, 1996;Similä and Tikka, 2006). Bayesian methods offer another approach (Brown et al, 2002), which is theoretically sound but may be a bit technical from a practical point of view.…”