This paper considers the problem of optimizing the ratio Tr[V T AV ]/Tr[V T BV ] over all unitary matrices V with p columns, where A, B are two positive definite matrices. This problem is common in supervised learning techniques. However, because its numerical solution is typically expensive it is often replaced by the simpler optimization problem which consists of optimizing Tr[V T AV ] under the constraint that V T BV = I, the identity matrix. The goal of this paper is to examine this trace ratio optimization problem in detail, to consider different algorithms for solving it, and to illustrate the use of these algorithms for dimensionality reduction. Introduction.A number of techniques in machine learning and high-dimensional data analysis are based on optimizing a trace of the form Tr [V T AV ] under certain constraints on V . This defines a projector with the basis V , which is then used for various dimensionality reduction tasks.A particular case of this scenario is the well-known Fisher linear discriminant analysis (LDA) [4]. The method, which is a prototypical approach of supervised learning, defines a linear hyperplane which best separates two or more datasets. This is achieved by trying to maximize the ratio of two traces. The first of these (the numerator) represents the in-between scatter, which measures how well the classes are separated in the projected space. The second (the denominator) represents the within scatter, which measures how well clustered each class is in the projected space.Consider the illustration shown in Figure 1. We are given n points which lie on a two-dimensional plane, with each point being labeled either as a triangle or as a square. Our goal is to find a good one-dimensional projection of these points. Specifically, we would like the projector to separate as best as possible the two distinct "classes," i.e., to separate triangles from squares. The left side of the figure shows a

No abstract

No abstract