The Lanczos method is often used to solve a large scale symmetric matrix eigenvalue problem. It is well-known that the single-vector Lanczos method can only find one copy of any multiple eigenvalue and encounters slow convergence towards clustered eigenvalues. On the other hand, the block Lanczos method can compute all or some of the copies of a multiple eigenvalue and, with a suitable block size, also compute clustered eigenvalues much faster. The existing convergence theory due to Saad for the block Lanczos method, however, does not fully reflect this phenomenon since the theory was established to bound approximation errors in each individual approximate eigenpairs. Here, it is argued that in the presence of an eigenvalue cluster, the entire approximate eigenspace associated with the cluster should be considered as a whole, instead of each individual approximate eigenvectors, and likewise for approximating clusters of eigenvalues. In this paper, we obtain error bounds on approximating eigenspaces and eigenvalue clusters. Our bounds are much sharper than the existing ones and expose true rates of convergence of the block Lanczos method towards eigenvalue clusters. Furthermore, their sharpness is independent of the closeness of eigenvalues within a cluster. Numerical examples are presented to support our claims. Also a possible extension to the generalized eigenvalue problem is outlined.

We first provide existence and uniqueness conditions for the solvability of an algebraic eigenvalue problem with eigenvector nonlinearity. We then present a local and global convergence analysis for a self-consistent field (SCF) iteration for solving the problem. The well-known sin \Theta theorem in the perturbation theory of Hermitian matrices plays a central role. The near-optimality of the local convergence rate of the SCF iteration revealed in this paper is demonstrated by examples from the discrete Kohn-Sham eigenvalue problem in electronic structure calculations and the maximization of the trace ratio in the linear discriminant analysis for dimension reduction.

The so-called Trust-Region Subproblem gets its name in the trust-region method in optimization and also plays a vital role in various other applications. Several numerical algorithms have been proposed in the literature for solving small-to-medium size dense problems as well as for large scale sparse problems. The Generalized Lanczos Trust-Region (GLTR) method proposed by [Gould, Lucidi, Roma and Toint, SIAM J. Optim., 9:504-525 (1999)] is a natural extension of the classical Lanczos method for the linear system to the trust-region subproblem. In this paper, we first analyze the convergence of GLTR to reveal its convergence behavior in theory and then propose new stopping criteria that can be integrated into GLTR for better numerical performance. Specifically, we develop a priori upper bounds for the convergence to both the optimal objective value as well as the optimal solution, and argue that these bounds can be efficiently estimated numerically and serve as stopping criteria for iterative methods such as GLTR. Two sets of numerical tests are presented. In the first set, we demonstrate the sharpness of the upper bounds, and for the second set, we integrate the upper bound estimate into the Fortran routine GLTR in the library GALAHAD as new stopping criteria, and test the trust-region solver TRU on the problem collection CUTEr. The numerical results show that, with the new stopping criteria in GLTR, the overall performance of TRU can be improved considerably.

No abstract

Abstract. Linear Discriminant Analysis (LDA) is one of the most popular approaches for feature extraction and dimension reduction to overcome the curse of the dimensionality of the high-dimensional data in many applications of data mining, machine learning, and bioinformatics. In this paper, we made two main contributions to an important LDA scheme, the generalized Foley-Sammon transform (GFST [7,13], or a trace ratio model [28]) and its regularization (RGFST) which handles the undersampled problem that involves small samples size n but with high number of features N (N > n) and arises frequently in many modern applications. Our first main result is to establish an equivalent reduced model for the RGFST which effectively improves the computational overhead. The iteration method proposed in [28] is applied to solve the GFST or the reduced RGFST. It has been proven [28] that this iteration converges globally and fast convergence was observed numerically, but there is no theoretical analysis on the convergence rate thus far.Our second main contribution completes this important and missing piece by proving the quadratic convergence even under two kinds of inexact computations. Practical implementations including computational complexity and storage requirement are also discussed. Our experimental results on several real world data sets indicate the efficiency of the algorithm and the advantages of the GFST model in classification.