A feasible method for optimization with orthogonality constraints

Abstract

Abstract. Minimization with orthogonality constraints (e.g., X X = I) and/or spherical constraints (e.g., x 2 = 1) has wide applications in polynomial optimization, combinatorial optimization, eigenvalue problems, sparse PCA, p-harmonic flows, 1-bit compressive sensing, matrix rank minimization, etc. These problems are difficult because the constraints are not only non-convex but numerically expensive to preserve during iterations. To deal with these difficulties, we propose to use a Crank-Nicolson-like update scheme to preserve the constraints and based on it, develop curvilinear search algorithms with lower per-iteration cost compared to those based on projections and geodesics. The efficiency of the proposed algorithms is demonstrated on a variety of test problems. In particular, for the maxcut problem, it exactly solves a decomposition formulation for the SDP relaxation. For polynomial optimization, nearest correlation matrix estimation and extreme eigenvalue problems, the proposed algorithms run very fast and return solutions no worse than those from their state-of-the-art algorithms. For the quadratic assignment problem, a gap 0.842% to the best known solution on the largest problem "tai256c" in QAPLIB can be reached in 5 minutes on a typical laptop.

An improved algorithm for unsupervised decomposition of a multi‐author document

“…Constrained optimization problems with such constraints have been addressed in the literature, in particular (Wen & Yin, 2013). The gradient descent search therein will find a critical point of Θ and guarantee that the matrix produced at each step is orthogonal.…”

“…If i≥maxNumSteps or ||A(i)X(i)'||F<ε, then return X(i)' and terminate, otherwise go to step II. The second condition checks whether the first order Lagrange optimality condition is close enough to being satisfied at X(i)' as discussed in Lemma 1 of (Wen & Yin, 2013). If so, then X(i)' is regarded as an approximate critical point of Θ and the algorithm terminates.…”

“…For the stopping parameters ε and maxNumSteps, values of 0.00001 and 1000 were used, the same ones used by (Wen & Yin, 2013) in their experiments. The line search algorithm is called "backtracking Armijo" and is described by Procedure 3.1 in (Nocedal & Wright, 1999).…”

“…The "Repeat until" condition is drawn from (26a) and Lemma 3 part 3 in (Wen & Yin, 2013). It is based on the Armijo condition which, if true, implies that the step length yields a sufficient decrease in Θ.…”

MERIT: A monotonically error-bound improving technique for unimodular quadratic programming

“…Next, suppose that we have to increase [18] with Barzilai-Borwein (BB) step size, and MERIT: (top) the UQP objective; (bottom) the required time for solving an UQP (n = 10) with same initialization.…”