The Trace Ratio Optimization Problem for Dimensionality Reduction

Ngo, Thanh Trung; Bellalij, M.; Saad, Yousef

doi:10.1137/090776603

Cited by 39 publications

(29 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this subsection, we use another approach to answer questions Q1 and Q2, which is inspired by J.-G. Sun's technique, see e.g., [8,16,17,18]. (1), P * = V * V H * , g be given by (12), and…”

Section: Approach Twomentioning

confidence: 99%

“…Perhaps, the most well-known NEPv of the form (1) is the discretized Kohn-Sham (KS) equation arising from density function theory in electronic structure calculations (see [3,11,14] and references therein). NEPv (1) also arises from the trace ratio optimization in the linear discriminant analysis for dimension reduction [12,20,21], and the Gross-Pitaevskii equation for modeling particles in the state of matter called the Bose-Einstein condensate [1,5,6]. We believe that more potential applications will emerge.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Perturbation analysis of an eigenvector-dependent nonlinear eigenvalue problem with applications

2019

View full text Add to dashboard Cite

The eigenvector-dependent nonlinear eigenvalue problem (NEPv) A(P )V = V Λ, where the columns of V ∈ C n×k are orthonormal, P = V V H , A(P ) is Hermitian, and Λ = V H A(P )V , arises in many important applications, such as the discretized Kohn-Sham equation in electronic structure calculations and the trace ratio problem in linear discriminant analysis. In this paper, we perform a perturbation analysis for the NEPv, which gives upper bounds for the distance between the solution to the original NEPv and the solution to the perturbed NEPv. A condition number for the NEPv is introduced, which reveals the factors that affect the sensitivity of the solution. Furthermore, two computable error bounds are given for the NEPv, which can be used to measure the quality of an approximate solution. The theoretical results are validated by numerical experiments for the Kohn-Sham equation and the trace ratio optimization.

show abstract

Section: Approach Twomentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Perturbation analysis of an eigenvector-dependent nonlinear eigenvalue problem with applications

2019

View full text Add to dashboard Cite

show abstract

“…In particular, we consider trace-ratio optimisation of matrices and compare them to existing trace-ratio optimisation procedures that have been used for MDA. We note that other procedures for optimising the trace-ratio exist [15, 16]. However, none of these procedures incorporate the cross-product of Stiefel manifolds structure of matrices presently considered.…”

Section: Introductionmentioning

confidence: 99%

Rigorous optimisation of multilinear discriminant analysis with Tucker and PARAFAC structures

2018

View full text Add to dashboard Cite

BackgroundWe propose rigorously optimised supervised feature extraction methods for multilinear data based on Multilinear Discriminant Analysis (MDA) and demonstrate their usage on Electroencephalography (EEG) and simulated data. While existing MDA methods use heuristic optimisation procedures based on an ambiguous Tucker structure, we propose a rigorous approach via optimisation on the cross-product of Stiefel manifolds. We also introduce MDA methods with the PARAFAC structure. We compare the proposed approaches to existing MDA methods and unsupervised multilinear decompositions.ResultsWe find that manifold optimisation substantially improves MDA objective functions relative to existing methods and on simulated data in general improve classification performance. However, we find similar classification performance when applied to the electroencephalography data. Furthermore, supervised approaches substantially outperform unsupervised mulitilinear methods whereas methods with the PARAFAC structure perform similarly to those with Tucker structures. Notably, despite applying the MDA procedures to raw Brain-Computer Interface data, their performances are on par with results employing ample pre-processing and they extract discriminatory patterns similar to the brain activity known to be elicited in the investigated EEG paradigms.ConclusionThe proposed usage of manifold optimisation constitutes the first rigorous and monotonous optimisation approach for MDA methods and allows for MDA with the PARAFAC structure. Our results show that MDA methods applied to raw EEG data can extract discriminatory patterns when compared to traditional unsupervised multilinear feature extraction approaches, whereas the proposed PARAFAC structured MDA models provide meaningful patterns of activity.Electronic supplementary materialThe online version of this article (10.1186/s12859-018-2188-0) contains supplementary material, which is available to authorized users.

show abstract

“…1 However, this problem also arises as a trace ratio maximization problem in linear discriminant analysis for dimension reduction. See [14], [24] and [25] for more on this application. The self-consistent field (SCF) iteration consists of computing iterates satisfying the linear eigenvalue problem…”

mentioning

confidence: 99%

A density matrix approach to the convergence of the self-consistent field iteration

Upadhyaya¹,

Jarlebring²,

Rubensson³

2021

NACO

View full text Add to dashboard Cite

In this paper, we present a local convergence analysis of the selfconsistent field (SCF) iteration using the density matrix as the state of a fixedpoint iteration. Conditions for local convergence are formulated in terms of the spectral radius of the Jacobian of a fixed-point map. The relationship between convergence and certain properties of the problem is explored by deriving upper bounds expressed in terms of higher gaps. This gives more information regarding how the gaps between eigenvalues of the problem affect the convergence, and hence these bounds are more insightful on the convergence behaviour than standard convergence results. We also provide a detailed analysis to describe the difference between the bounds and the exact convergence factor for an illustrative example. Finally we present numerical examples and compare the exact value of the convergence factor with the observed behaviour of SCF, along with our new bounds and the characterization using the higher gaps. We provide heuristic convergence factor estimates in situations where the bounds fail to well capture the convergence.

show abstract

The Trace Ratio Optimization Problem for Dimensionality Reduction

Cited by 39 publications

References 6 publications

Perturbation analysis of an eigenvector-dependent nonlinear eigenvalue problem with applications

Perturbation analysis of an eigenvector-dependent nonlinear eigenvalue problem with applications

Rigorous optimisation of multilinear discriminant analysis with Tucker and PARAFAC structures

A density matrix approach to the convergence of the self-consistent field iteration

Contact Info

Product

Resources

About