A Relationship between Linear Discriminant Analysis and the Generalized Minimum Squared Error Solution

Park, Cheong Hee; Park, Haesun

doi:10.1137/040607599

Cited by 41 publications

(27 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such a W * is identical with G in (16). Consequently, we have found a relationship between the ridge estimation problem in (18) and the RLDA problem in (6).…”

Section: Relationships Between Rfda and Ridge Regressionmentioning

confidence: 68%

“…Recently, similar relationships have been studied for multi-class (c > 2) problems [7,16,20]. In particular, Park and Park [16] proposed an efficient algorithm for LDA via a least mean squared error procedure in the multi-class problem.…”

Section: Linear Discriminant Analysismentioning

confidence: 99%

“…6: end procedure between LDA and a least mean squared error procedure in multi-class (c > 2) problems were discussed by [7,16,20].…”

Section: Relationships Between Rfda and Ridge Regressionmentioning

confidence: 99%

“…It is of great interest to obtain a similar relationship in multi-class problems. A significant literature has emerged to address this issue [7,16,20]. However, the results obtained by these authors are subject to restrictive conditions.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis

Zhang

Dai

Jordan

2009

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

Abstract. Fisher linear discriminant analysis (LDA) and its kernel extensionkernel discriminant analysis (KDA)-are well known methods that consider dimensionality reduction and classification jointly. While widely deployed in practical problems, there are still unresolved issues surrounding their efficient implementation and their relationship with least mean squared error procedures. In this paper we address these issues within the framework of regularized estimation. Our approach leads to a flexible and efficient implementation of LDA as well as KDA. We also uncover a general relationship between regularized discriminant analysis and ridge regression. This relationship yields variations on conventional LDA based on the pseudoinverse and a direct equivalence to an ordinary least squares estimator. Experimental results on a collection of benchmark data sets demonstrate the effectiveness of our approach.

show abstract

“…Such a W * is identical with G in (16). Consequently, we have found a relationship between the ridge estimation problem in (18) and the RLDA problem in (6).…”

Section: Relationships Between Rfda and Ridge Regressionmentioning

confidence: 68%

Section: Linear Discriminant Analysismentioning

confidence: 99%

“…6: end procedure between LDA and a least mean squared error procedure in multi-class (c > 2) problems were discussed by [7,16,20].…”

Section: Relationships Between Rfda and Ridge Regressionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis

Zhang

Dai

Jordan

2009

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

show abstract

“…The optimal W is given by [Hastie et al 2001] ( 9) which is determined by X̃ and Ỹ. Both Y 1 and Y 2 defined above, as well as the one in [Park and Park 2005], could be used to define the centered indicator matrix Ỹ. However, the resulting linear regression models using these indicator matrices are not, in general, equivalent to LDA.…”

Section: Multivariate Linear Regression With a Class Indicator Matrixmentioning

confidence: 99%

Developmental stage annotation of Drosophila gene expression pattern images via an entire solution path for LDA

Chen

Janardan

et al. 2008

ACM Trans. Knowl. Discov. Data

View full text Add to dashboard Cite

Gene expression in a developing embryo occurs in particular cells (spatial patterns) in a time-specific manner (temporal patterns), which leads to the differentiation of cell fates. Images of a Drosophila melanogaster embryo at a given developmental stage, showing a particular gene expression pattern revealed by a gene-specific probe, can be compared for spatial overlaps. The comparison is fundamentally important to formulating and testing gene interaction hypotheses. Expression pattern comparison is most biologically meaningful when images from a similar time point (developmental stage) are compared. In this paper, we present LdaPath, a novel formulation of Linear Discriminant Analysis (LDA) for automatic developmental stage range classification. It employs multivariate linear regression with the L 1 -norm penalty controlled by a regularization parameter for feature extraction and visualization. LdaPath computes an entire solution path for all values of regularization parameter with essentially the same computational cost as fitting one LDA model. Thus, it facilitates efficient model selection. It is based on the equivalence relationship between LDA and the least squares method for multi-class classifications. This equivalence relationship is established under a mild condition, which we show empirically to hold for many high-dimensional datasets, such as expression pattern images. Our experiments on a collection of 2705 expression pattern images show the effectiveness of the proposed algorithm. Results also show that the LDA model resulting from LdaPath is sparse, and irrelevant features may be removed. Thus, LdaPath provides a general framework for simultaneous feature selection and feature extraction.

show abstract

Discriminant Analysis for Dimensionality Reduction: An Overview of Recent Developments

Biometrics

View full text Add to dashboard Cite

A Relationship between Linear Discriminant Analysis and the Generalized Minimum Squared Error Solution

Cited by 41 publications

References 16 publications

A Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis

A Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis

Developmental stage annotation of Drosophila gene expression pattern images via an entire solution path for LDA

Discriminant Analysis for Dimensionality Reduction: An Overview of Recent Developments

Contact Info

Product

Resources

About