Input selection and shrinkage in multiresponse linear regression

Abstract

The regression problem of modeling several response variables using the same set of input variables is considered. The model is linearly parameterized and the parameters are estimated by minimizing the error sum of squares subject to a sparsity constraint. The constraint has the effect of eliminating useless inputs and constraining the parameters of the remaining inputs in the model. Two algorithms for solving the resulting convex cone programming problem are proposed. The first algorithm gives a pointwise solution, while the second one computes the entire path of solutions as a function of the constraint parameter. Based on experiments with real data sets, the proposed method has a similar performance to existing methods. In simulation experiments, the proposed method is competitive both in terms of prediction accuracy and correctness of input selection. The advantages become more apparent when many correlated inputs are available for model construction.

An efficient projection for l 1 , ∞ regularization

“…This norm penalizes the sum of the l 2 norms of each row of W (Yuan & Lin, 2006;Meier et al, 2006;Similä & Tikka, 2007;Park & Hastie, 2006;Obozinski et al, 2006;Argyriou et al, 2007;Schmidt et al, 2009). 3. A projected gradient method for l 1,∞ regularization…”

Sparse multinomial kernel discriminant analysis (sMKDA)

“…Inducing sparsity as a consequence of optimization might, therefore, prove beneficial and the adoption of a multivariable generalization of, say, the LASSO algorithm [e.g. 44] or the method based on surrogate optimization described in [18]. This would have the advantage of performing regularization directly but would add to the cross-validation burden to select the degree of regularization required.…”

A locally weighted learning method based on a data gravitation model for multi-target regression

“…Spyromitros-Xioufis et al 38 analysed how several multi-label approaches, such as the binary relevance, stacked generalization and classifier chains, are straightforward of applying in multi-target regression contexts. As for algorithm adaptation category, a large number of methods have been proposed, such as statistical methods 37 , support vector machines 42,17 , kernel approaches 3 , multi-target regression trees 25 , and rule-based methods 1 .…”