Convex envelopes for fixed rank approximation

Андерссон, Фредрик; Carlsson, Marcus

doi:10.1007/s11590-017-1146-5

Cited by 18 publications

(33 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, rank(X) and the nuclear norm are quite far apart and the method leads to a bias in the solution, which led the authors of [20] to suggest working instead with the convex envelope of rank(X) + 1 2 X − D 2 F for which they obtained an explicit expression. They also provided the convex envelope when rank(X) is replaced by the indicator functional of the set {X : rank(X) ≤ K}, in order to treat problems where a matrix of a fixed rank is sought, and this convex envelope was further studied in [1].…”

Section: Outline and Motivationmentioning

confidence: 99%

On Convex Envelopes and Regularization of Non-convex Functionals Without Moving Global Minima

Carlsson¹

2019

J Optim Theory Appl

Self Cite

View full text Add to dashboard Cite

We provide theory for the computation of convex envelopes of non-convex functionals including an 2 -term, and use these to suggest a method for regularizing a more general set of problems. The applications are particularly aimed at compressed sensing and low rank recovery problems but the theory relies on results which potentially could be useful also for other types of non-convex problems. For optimization problems where the 2 -term contains a singular matrix we prove that the regularizations never move the global minima. This result in turn relies on a theorem concerning the structure of convex envelopes which is interesting in its own right. It says that at any point where the convex envelope does not touch the non-convex functional we necessarily have a direction in which the convex envelope is affine.

show abstract

Section: Outline and Motivationmentioning

confidence: 99%

On Convex Envelopes and Regularization of Non-convex Functionals Without Moving Global Minima

Carlsson¹

2019

J Optim Theory Appl

Self Cite

View full text Add to dashboard Cite

show abstract

“…To this end, we develop a generic nested binary search algorithm, which in each iteration solves a simple problem. While for well-known low-rank inducing norms such as the nuclear norm [38] and the low-rank inducing Frobenius norm [1,14,29,30], our algorithm will recover the same efficiency, for other norms such as the low-rank inducing spectral norm [46], our approach improves the computational complexity significantly, especially in the vector-valued case. Finally, [45] proposes a non-analytic approach for an extended class of not necessarily unitarily invariant lowrank inducing norms (see [27]).…”

Section: Introductionmentioning

confidence: 82%

Efficient Proximal Mapping Computation for Low-Rank Inducing Norms

Grussler

Giselsson

2021

J Optim Theory Appl

View full text Add to dashboard Cite

Low-rank inducing unitarily invariant norms have been introduced to convexify problems with a low-rank/sparsity constraint. The most well-known member of this family is the so-called nuclear norm. To solve optimization problems involving such norms with proximal splitting methods, efficient ways of evaluating the proximal mapping of the low-rank inducing norms are needed. This is known for the nuclear norm, but not for most other members of the low-rank inducing family. This work supplies a framework that reduces the proximal mapping evaluation into a nested binary search, in which each iteration requires the solution of a much simpler problem. The simpler problem can often be solved analytically as demonstrated for the so-called low-rank inducing Frobenius and spectral norms. The framework also allows to compute the proximal mapping of increasing convex functions composed with these norms as well as projections onto their epigraphs.

show abstract

“…, where √ • denotes the square root of matrix. The trace-norm trace(•) is proven to be the convex envelope of the matrix rank (Andersson, Carlsson, and Olsson 2017).…”

Section: Functional Regularizationmentioning

confidence: 99%

Adaptive Activation Network and Functional Regularization for Efficient and Flexible Deep Multi-Task Learning

Liu

Yang

Xie

et al. 2020

AAAI

View full text Add to dashboard Cite

Multi-task learning (MTL) is a common paradigm that seeks to improve the generalization performance of task learning by training related tasks simultaneously. However, it is still a challenging problem to search the flexible and accurate architecture that can be shared among multiple tasks. In this paper, we propose a novel deep learning model called Task Adaptive Activation Network (TAAN) that can automatically learn the optimal network architecture for MTL. The main principle of TAAN is to derive flexible activation functions for different tasks from the data with other parameters of the network fully shared. We further propose two functional regularization methods that improve the MTL performance of TAAN. The improved performance of both TAAN and the regularization methods is demonstrated by comprehensive experiments.

show abstract

Convex envelopes for fixed rank approximation

Cited by 18 publications

References 13 publications

On Convex Envelopes and Regularization of Non-convex Functionals Without Moving Global Minima

On Convex Envelopes and Regularization of Non-convex Functionals Without Moving Global Minima

Efficient Proximal Mapping Computation for Low-Rank Inducing Norms

Adaptive Activation Network and Functional Regularization for Efficient and Flexible Deep Multi-Task Learning

Contact Info

Product

Resources

About