Exact Penalty Function for $\ell_{2,1}$ Norm Minimization over the Stiefel Manifold

Nachuan, Xiao,; Liu, Xin; Yuan, Ya-xiang

doi:10.1137/20m1354313

Cited by 11 publications

(12 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, a novel class of efficient approaches for Stiefel manifold optimization are developed based on exact penalty models. Inspired by the exact penalty model (PenC) for smooth optimization over the Stiefel manifold [61,35], [62] extends PenC to 2,1 -norm regularized cases and proposes a proximal gradient method called PenCPG. In PenCPG, the proximal subproblem has a closed-form solution, which leads to its numerical superiority over existing Riemannian proximal gradient approaches.…”

Section: Proximal Gradient Methodsmentioning

confidence: 99%

“…Therefore, the objective functions for the applications mentioned in [2,15,41,54,14,62,63] are Whitney C 1 -stratifiable. As a result, we can develop subgradient methods to minimize those functions and prove their convergence properties based on the framework presented in [23].…”

Section: Lemma 44 ([7]mentioning

confidence: 99%

“…Our interest in OCP with nonsmooth objective functions comes from its various applications in different engineering fields, including sparse principal component analysis [15,62], robust subspace recovery [41,57], packing problems [2], and training orthogonally constrained neural networks [3,5], where the objective function is only locally Lipschitz continuous and usually not weakly convex (i.e. f (X) + τ 2 X 2 F is convex for some constant τ ≥ 0 [19]).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Constraint Dissolving Approach for Nonsmooth Optimization over the Stiefel Manifold

Hu¹,

Nachuan²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

This paper focus on the minimization of a possibly nonsmooth objective function over the Stiefel manifold. The existing approaches either lack efficiency or can only tackle prox-friendly objective functions. We propose a constraint dissolving function named NCDF and show that it has the same first-order stationary points and local minimizers as the original problem in a neighborhood of the Stiefel manifold. Furthermore, we show that the Clarke subdifferential of NCDF is easy to achieve from the Clarke subdifferential of the objective function. Therefore, various existing approaches for unconstrained nonsmooth optimization can be directly applied to nonsmooth optimization problems on the Stiefel manifold. We propose a framework for developing subgradient-based methods and establish their convergence properties based on prior works. Preliminary numerical experiments further highlight that the proposed constraint dissolving approach enables the efficient and direct implementations of various unconstrained solvers to nonsmooth optimization problems over the Stiefel manifold.

show abstract

Section: Proximal Gradient Methodsmentioning

confidence: 99%

Section: Lemma 44 ([7]mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Constraint Dissolving Approach for Nonsmooth Optimization over the Stiefel Manifold

Hu¹,

Nachuan²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For optimization problems on the Stiefel manifold, i.e. M = S m,s := {X ∈ R m×s : X ⊤ X = I s }, [63] presents an exact penalty model named PenC based on the explicit expression of the multipliers [22], which further yields efficient infeasible algorithms [63,64,31,65]. However, PenC involves ∇ f in its objective function as well.…”

Section: Existing Approachesmentioning

confidence: 99%

Dissolving Constraints for Riemannian Optimization

Nachuan¹,

Liu²,

Toh³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a class of constraint dissolving approaches for optimization problems over closed Riemannian manifolds. In these proposed approaches, solving a Riemannian optimization problem is transferred into the unconstrained minimization of a constraint dissolving function named CDF. Different from existing exact penalty functions, the exact gradient and Hessian of CDF are easy to compute. We study the theoretical properties of CDF and prove that the original problem and CDF have the same first-order and second-order stationary points, local minimizers, and Łojasiewicz exponents in a neighborhood of the feasible region. Remarkably, the convergence properties of our proposed constraint dissolving approaches can be directly inherited from the existing rich results in unconstrained optimization. Therefore, the proposed constraint dissolving approaches build up short cuts from unconstrained optimization to Riemannian optimization. Several illustrative examples further demonstrate the potential of our proposed constraint dissolving approaches.

show abstract

“…In this section, we apply the error bounds established in the last section to study exact penalties for optimization problem (1) with nonnegative orthogonality constraints. Our exact penalty results only require (local) Lipschitz continuity of F , and they can be applied to nonsmooth optimization, for example, F involving a group sparse regularization term [24].…”

Section: Exact Penalties For Nonnegative Orthogonality Constraintsmentioning

confidence: 99%

Tight Error Bounds for Nonnegative Orthogonality Constraints and Exact Penalties

Chen¹,

He²,

Zhang³

2022

Preprint

View full text Add to dashboard Cite

For the intersection of the Stiefel manifold and the set of nonnegative matrices in R n×r , we present global and local error bounds with easily computable residual functions and explicit coefficients. Moreover, we show that the error bounds cannot be improved except for the coefficients, which explains why two square-root terms are necessary in the bounds when 1 < r < n for the nonnegativity and orthogonality, respectively. The error bounds are applied to penalty methods for minimizing a Lipschitz continuous function with nonnegative orthogonality constraints. Under only the Lipschitz continuity of the objective function, we prove the exactness of penalty problems that penalize the nonnegativity constraint, or the orthogonality constraint, or both constraints. Our results cover both global and local minimizers.

show abstract

Exact Penalty Function for $\ell_{2,1}$ Norm Minimization over the Stiefel Manifold

Cited by 11 publications

References 27 publications

A Constraint Dissolving Approach for Nonsmooth Optimization over the Stiefel Manifold

A Constraint Dissolving Approach for Nonsmooth Optimization over the Stiefel Manifold

Dissolving Constraints for Riemannian Optimization

Tight Error Bounds for Nonnegative Orthogonality Constraints and Exact Penalties

Contact Info

Product

Resources

About