Generic MANOVA limit theorems for products of projections

Abstract

We study the convergence of the empirical spectral distribution of ABA for N × N orthogonal projection matrices A and B, where 1 N Tr(A) and 1 N Tr(B) converge as N → ∞, to Wachter's MANOVA law. Using free probability, we show mild sufficient conditions for convergence in moments and in probability, and use this to prove a conjecture of Haikin, Zamir, and Gavish (2017) on random subsets of unit-norm tight frames. This result generalizes previous ones of Farrell (2011) andMagsino, Mixon, andParshall (2021). We also derive an explicit recursion for the difference between the empirical moments 1 N Tr((ABA) k ) and the limiting MANOVA moments, and use this to prove a sufficient condition for convergence in probability of the largest eigenvalue of ABA to the right edge of the support of the limiting law in the special case where that law belongs to the Kesten-McKay family. As an application, we give a new proof of convergence in probability of the largest eigenvalue when B is unitarily invariant; equivalently, this determines the limiting operator norm of a rectangular submatrix of size 1 2 N × αN of a Haar-distributed N × N unitary matrix for any α ∈ (0, 1). Unlike previous proofs, we use only moment calculations and non-asymptotic bounds on the unitary Weingarten function, which we believe should pave the way to analyzing the largest eigenvalue for products of random projections having other distributions.