Dimensionality reduction approach to multivariate prediction

Abraham, Bovas; Merola, Giovanni Maria

doi:10.1016/j.csda.2003.11.021

Cited by 25 publications

(15 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…), PLS produces the same result as RRR. Alternative methods can be constructed by tuning the objective function between var(Xu i ) and corr 2 (Xu i , y) [1]. However, these methods require an additional parameter and are thus not considered here.…”

Section: Partial Least Squares (Pls)mentioning

confidence: 99%

See 1 more Smart Citation

Local Dimensionality Reduction for Non-Parametric Regression

Hoffmann

Schaal

Vijayakumar

2009

Neural Process Lett

View full text Add to dashboard Cite

Locally-weighted regression is a computationally-efficient technique for non-linear regression. However, for high-dimensional data, this technique becomes numerically brittle and computationally too expensive if many local models need to be maintained simultaneously. Thus, local linear dimensionality reduction combined with locally-weighted regression seems to be a promising solution. In this context, we review linear dimensionalityreduction methods, compare their performance on non-parametric locally-linear regression, and discuss their ability to extend to incremental learning. The considered methods belong to the following three groups: (1) reducing dimensionality only on the input data, (2) modeling the joint input-output data distribution, and (3) optimizing the correlation between projection directions and output data. Group 1 contains principal component regression (PCR); group 2 contains principal component analysis (PCA) in joint input and output space, factor analysis, and probabilistic PCA; and group 3 contains reduced rank regression (RRR) and partial least squares (PLS) regression. Among the tested methods, only group 3 managed to achieve robust performance even for a non-optimal number of components (factors or projection directions). In contrast, group 1 and 2 failed for fewer components since these methods rely on the correct estimate of the true intrinsic dimensionality. In group 3, PLS is the only method for which a computationally-efficient incremental implementation exists.

show abstract

Section: Partial Least Squares (Pls)mentioning

confidence: 99%

“…SIMPLS maximizes (u T i X T y) 2 under the constraint that the projections Xu i are orthogonal to each other, and that u T i u i = 1. Since (u T i X T y) 2 = corr 2 (Xu i , y) var(Xu i ), PLS has therefore been considered as a mixture between RRR and PCR [1]. For sphericallydistributed input data (var(Xu i ) = const.…”

Section: Partial Least Squares (Pls)mentioning

confidence: 99%

Local Dimensionality Reduction for Non-Parametric Regression

Hoffmann

Schaal

Vijayakumar

2009

Neural Process Lett

View full text Add to dashboard Cite

show abstract

“…In the past decades, CCA and its variants have been successfully used in many research areas such as facial expression recognition [3], image analysis [4], position estimation of robots [5], parameter estimation of posture [6], data regression analysis [7], image texture analysis [8], image retrieval [9], content based text mining [10] and asymptotic convergence of the functions [11]. Given a data set with two views X and Y , the goal of CCA is to seek a set of basis vector pairs which would maximize the correlation of the two views when been projected into lower dimensional space.…”

mentioning

confidence: 99%

A New Canonical Correlation Analysis Algorithm with Local Discrimination

Yan

Zhang

2009

Neural Process Lett

View full text Add to dashboard Cite

In this paper, a new feature extraction algorithm is developed based on canonical correlation analysis (CCA), called Local Discrimination CCA (LDCCA). The method considers a combination of local properties and discrimination between different classes. Not only the correlations between sample pairs but also the correlations between samples and their local neighborhoods are taken into consideration in LDCCA. Effective class separation is achieved by maximizing local within-class correlations and minimizing local between-class correlations simultaneously. Besides, a kernel version of LDCCA (KLDCCA) is proposed to cope with nonlinear problems in experiments. The experimental results on an artificial dataset, multiple feature databases and face databases including ORL, Yale, AR validate the effectiveness of the proposed methods.

show abstract

“…Highly correlated inputs cause the problem of collinearity: model interpretation is misleading as the importance of an input in the model can be compensated by another input. Traditional methods for meeting these problems are pure input selection (Sparks et al, 1985), regularization or shrinking (Breiman and Friedman, 1997;Srivastava and Solanky, 2003), and subspace methods (Abraham and Merola, 2005). Shrinking means that the regression coefficients are constrained such that the unimportant inputs tend to have coefficient values close to zero.…”

Section: Introductionmentioning

confidence: 99%

Input selection and shrinkage in multiresponse linear regression

Similä¹,

Tikka²

2007

Computational Statistics & Data Analysis

View full text Add to dashboard Cite

The regression problem of modeling several response variables using the same set of input variables is considered. The model is linearly parameterized and the parameters are estimated by minimizing the error sum of squares subject to a sparsity constraint. The constraint has the effect of eliminating useless inputs and constraining the parameters of the remaining inputs in the model. Two algorithms for solving the resulting convex cone programming problem are proposed. The first algorithm gives a pointwise solution, while the second one computes the entire path of solutions as a function of the constraint parameter. Based on experiments with real data sets, the proposed method has a similar performance to existing methods. In simulation experiments, the proposed method is competitive both in terms of prediction accuracy and correctness of input selection. The advantages become more apparent when many correlated inputs are available for model construction.

show abstract

Dimensionality reduction approach to multivariate prediction

Cited by 25 publications

References 18 publications

Local Dimensionality Reduction for Non-Parametric Regression

Local Dimensionality Reduction for Non-Parametric Regression

A New Canonical Correlation Analysis Algorithm with Local Discrimination

Input selection and shrinkage in multiresponse linear regression

Contact Info

Product

Resources

About