From the simplex to the sphere: Faster constrained optimization using the Hadamard parametrization

Li, Qiuwei; McKenzie, Daniel; Yin, Wotao

doi:10.48550/arxiv.2112.05273

Cited by 3 publications

(4 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• The eigenvalue example above (scenario (b)) essentially lifts the simplex to the sphere by entrywise squaring. This lift is the topic of [35]. We extend their results beyond the case of convex f , as a particular case of our general construction of lifts in Section…”

Section: Contributionsmentioning

confidence: 91%

“…We have already encountered them in Examples 2.14 and 2. 35, where we proved some of their properties using ad hoc arguments. They can be easily treated systematically using our framework, computing L and Q using the expressions (11) for embedded submanifolds.…”

Section: Curvesmentioning

confidence: 97%

“…By Proposition 2.42, the fiber product lift of the simplex is equivalent (for the purposes of checking our desirable properties) to the lift ϕ = ϕ • ψ sending a unit vector y ∈ S n−1 to y ⊙2 . Li et al [35] study this particular lift for optimization of (P) through (Q) for convex f . Example 4.5 (Torus to annulus).…”

Section: Lift Construction Via Fiber Products More Examplesmentioning

confidence: 99%

See 2 more Smart Citations

The effect of smooth parametrizations on nonconvex optimization landscapes

Levin¹,

Kileel²,

Boumal³

2022

Preprint

View full text Add to dashboard Cite

We develop new tools to study the landscapes of nonconvex optimization problems. Specifically, we leverage the fact that many such problems can be paired with others via a smooth parametrization of their domains. The global optima of such pairs of problems correspond to each other, but their landscapes can be significantly different. We introduce a framework to relate the two landscapes. Applications include: optimization over low-rank matrices and tensors by optimizing over a factorization; the Burer-Monteiro approach to semidefinite programs; training neural networks by optimizing over their weights and biases; and quotienting out symmetries. In all these examples, one of the two problems is smooth, and hence can be tackled with algorithms for optimization on manifolds. These can find desirable points (e.g., critical points). We determine the properties that ensure these map to desirable points for the other problem via the smooth parametrization. These new tools enable us to strengthen guarantees for an array of optimization problems, previously obtained on a case-bycase basis in the literature.

show abstract

Section: Contributionsmentioning

confidence: 91%

Section: Curvesmentioning

confidence: 97%

Section: Lift Construction Via Fiber Products More Examplesmentioning

confidence: 99%

See 1 more Smart Citation

The effect of smooth parametrizations on nonconvex optimization landscapes

Levin¹,

Kileel²,

Boumal³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Sparse least square regression with probabilistic simplex constraint. The authors of [52,33] consider the spherical constrained formulation of the following optimization problems:…”

Section: Motivating Examplesmentioning

confidence: 99%

A projected semismooth Newton method for a class of nonconvex composite programs with strong prox-regularity

Hu¹,

Deng²,

Wu³

et al. 2023

Preprint

View full text Add to dashboard Cite

This paper aims to develop a Newton-type method to solve a class of nonconvex composite programs. In particular, the nonsmooth part is possibly nonconvex. To tackle the nonconvexity, we develop a notion of strong prox-regularity which is related to the singleton property, Lipschitz continuity, and monotonicity of the associated proximal operator, and we verify it in various classes of functions, including weakly convex functions, indicator functions of proximally smooth sets, and two specific sphere-related nonconvex nonsmooth functions. In this case, the problem class we are concerned with covers smooth optimization problems on manifold and certain composite optimization problems on manifold. For the latter, the proposed algorithm is the first second-order type method. Combining with the semismoothness of the proximal operator, we design a projected semismooth Newton method to find a root of the natural residual induced by the proximal gradient method. Since the corresponding natural residual may not be globally monotone, an extra projection is added on the usual semismooth Newton step and new criteria are proposed for the switching between the projected semismooth Newton step and the proximal step. The global convergence is then established under the strong prox-regularity. Based on the BD regularity condition, we establish local superlinear convergence. Numerical experiments demonstrate the effectiveness of our proposed method compared with state-of-the-art ones.

show abstract

Anomaly detection in the probability simplex under different geometries

Legaria,

Mota,

Martinez

et al. 2023

Info. Geo.

View full text Add to dashboard Cite

An open problem in data science is that of anomaly detection. Anomalies are instances that do not maintain a certain property that is present in the remaining observations in a dataset. Several anomaly detection algorithms exist, since the process itself is ill-posed mainly because the criteria that separates common or expected vectors from anomalies are not unique. In the most extreme case, data is not labelled and the algorithm has to identify the vectors that are anomalous, or assign a degree of anomaly to each vector. The majority of anomaly detection algorithms do not make any assumptions about the properties of the feature space in which observations are embedded, which may affect the results when those spaces present certain properties. For instance, compositional data such as normalized histograms, that can be embedded in a probability simplex, constitute a particularly relevant case. In this contribution, we address the problem of detecting anomalies in the probability simplex, relying on concepts from Information Geometry, mainly by focusing our efforts in the distance functions commonly applied in that context. We report the results of a series of experiments and conclude that when a specific distance-based anomaly detection algorithm relies on Information Geometry-related distance functions instead of the Euclidean distance, the performance is significantly improved.

show abstract

From the simplex to the sphere: Faster constrained optimization using the Hadamard parametrization

Cited by 3 publications

References 22 publications

The effect of smooth parametrizations on nonconvex optimization landscapes

The effect of smooth parametrizations on nonconvex optimization landscapes

A projected semismooth Newton method for a class of nonconvex composite programs with strong prox-regularity

Anomaly detection in the probability simplex under different geometries

Contact Info

Product

Resources

About