Improving implementation of linear discriminant analysis for the high dimension/small sample size problem

Tebbens, Jurjen Duintjer; Schlesinger, Pavel

doi:10.1016/j.csda.2007.02.001

Cited by 32 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For large data this method is rather expensive as it requires O((n + g) 2 p) operations. The method proposed by [12] seems a better alternative if one needs to avoid the calculation of large B and W. Further related results can be found in [38].…”

Section: Basic Notations Definitions and Assumptions Of Classical Ldamentioning

confidence: 97%

Recipes for sparse LDA of horizontal data

Trendafilov

Gebru

2016

METRON

View full text Add to dashboard Cite

Many important modern applications require analyzing data with more variables than observations, called for short horizontal. In such situation the classical Fisher's linear discriminant analysis (LDA) does not possess solution because the within-group scatter matrix is singular. Moreover, the number of the variables is usually huge and the classical type of solutions (discriminant functions) are difficult to interpret as they involve all available variables. Nowadays, the aim is to develop fast and reliable algorithms for sparse LDA of horizontal data. The resulting discriminant functions depend on very few original variables, which facilitates their interpretation. The main theoretical and numerical challenge is how to cope with the singularity of the within-group scatter matrix. This work aims at classifying the existing approaches according to the way they tackle this singularity issue, and suggest new ones.

show abstract

Section: Basic Notations Definitions and Assumptions Of Classical Ldamentioning

confidence: 97%

Recipes for sparse LDA of horizontal data

Trendafilov

Gebru

2016

METRON

View full text Add to dashboard Cite

show abstract

Section: Sss Problem Of Lda At Global Levelmentioning

confidence: 99%

“…Owing to this SSS problem, LDA cannot be directly applied for the image clustering problem [12] wherein the SSS problem in image clustering occurs when the total number of images in an image dataset is less than the image feature dimension. Researchers have proposed various approaches including [13–16] to deal with the SSS problem of LDA at global level. Among them, Friedman proposed regularised discriminant analysis (RDA) [13] in which the identity matrix I with regularisation parameter λ > 0 was added to make S t non‐singular such as the following

A^{*} = \arg \max_{A} T_{r} ()(, {()(, S_{t} + λ bold-italicI)}^{- 1} S_{b})

In RDA, S b and S t are evaluated at global level in the whole image dataset X .…”

Section: Introductionmentioning

confidence: 99%

Image clustering using exponential discriminant analysis

Ahmed

2015

IET Computer Vision

View full text Add to dashboard Cite

Local learning based image clustering models are usually employed to deal with images sampled from the non-linear manifold. Recently, linear discriminant analysis (LDA) based various clustering models were proposed. However, in these clustering models, regularisation parameter was added to handle the small-sample-size (SSS) problem of LDA for highdimensional image data. Owing to this, the authors had to tune a number of clustering parameters for optimal clustering performance in existing local learning based clustering approaches. In this study, the less-parameterised local learning based image clustering model is proposed. The proposed local exponential discriminant clustering (LEDC) model is based on exponential discriminant analysis (EDA). In the LEDC model, local scatter matrices are projected in the exponential domain in order to handle the SSS problem of LDA without adding regularisation parameter. In the proposed LEDC model, k-nearest neighbours are the only clustering parameter as compared with existing local learning based clustering approaches such as normalised cut, spectral embedded clustering and local discriminant model and global integration (LDMGI). Experimental results on twelve benchmark image datasets show that the LEDC model achieved a comparable clustering performance as that of the near competitor LDMGI model. Clustering performance is comparable because no discriminant information of LDA is lost in EDA. The authors concluded that the proposed LEDC clustering model is less-parameterised with a comparable clustering performance as that of existing state-of-the-art local learning based clustering approaches.

show abstract

Template Matching Techniques in Computer Vision

Brunelli¹

2009

612

343

View full text Add to dashboard Cite

Improving implementation of linear discriminant analysis for the high dimension/small sample size problem

Cited by 32 publications

References 18 publications

Recipes for sparse LDA of horizontal data

Recipes for sparse LDA of horizontal data

Image clustering using exponential discriminant analysis

Template Matching Techniques in Computer Vision

Contact Info

Product

Resources

About