On the rotational invariant L1-norm PCA

Neumayer, Sebastian; Nimmer, Max; Setzer, Simon; Steidl, Gabriele

doi:10.1016/j.laa.2019.10.030

Cited by 9 publications

(6 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A proof can be found in [37,Thm. 6.1] along with an example which demonstrates the necessity of the Lipschitz continuity of f in the manifold setting in the first part of the theorem.…”

Section: Discussionmentioning

confidence: 99%

On the Robust PCA and Weiszfeld’s Algorithm

Neumayer

Nimmer

Setzer³

et al. 2019

Appl Math Optim

Self Cite

View full text Add to dashboard Cite

The principal component analysis (PCA) is a powerful standard tool for reducing the dimensionality of data. Unfortunately, it is sensitive to outliers so that various robust PCA variants were proposed in the literature. This paper addresses the robust PCA by successively determining the directions of lines having minimal Euclidean distances from the data points. The corresponding energy functional is not differentiable at a finite number of directions which we call anchor directions. We derive a Weiszfeld-like algorithm for minimizing the energy functional which has several advantages over existing algorithms. Special attention is paid to the careful handling of the anchor directions, where we take the relation between local minima and one-sided derivatives of Lipschitz continuous functions on submanifolds of R d into account. Using ideas for stabilizing the classical Weiszfeld algorithm at anchor points and the Kurdyka-Lojasiewicz property of the energy functional, we prove global convergence of the whole sequence of iterates generated by the algorithm to a critical point of the energy functional. Numerical examples demonstrate the very good performance of our algorithm.

show abstract

“…A proof can be found in [37,Thm. 6.1] along with an example which demonstrates the necessity of the Lipschitz continuity of f in the manifold setting in the first part of the theorem.…”

Section: Discussionmentioning

confidence: 99%

On the Robust PCA and Weiszfeld’s Algorithm

Neumayer

Nimmer

Setzer³

et al. 2019

Appl Math Optim

Self Cite

View full text Add to dashboard Cite

show abstract

“…For the Euclidean norm and h(x) = x 2 we get the classical PCA approach and for h(x) = x the robust rotationally invariant L 1 -norm PCA, recently discussed in [30,35].…”

Section: Example 63 Let Us Specify Two Special Cases Of Ppnns With One Layermentioning

confidence: 99%

Parseval Proximal Neural Networks

Hasannasab

Hertrich

Neumayer

et al. 2020

J Fourier Anal Appl

Self Cite

View full text Add to dashboard Cite

The aim of this paper is twofold. First, we show that a certain concatenation of a proximity operator with an affine operator is again a proximity operator on a suitable Hilbert space. Second, we use our findings to establish so-called proximal neural networks (PNNs) and stable tight frame proximal neural networks. Let $$\mathcal {H}$$ H and $$\mathcal {K}$$ K be real Hilbert spaces, $$b \in \mathcal {K}$$ b ∈ K and $$T \in \mathcal {B} (\mathcal {H},\mathcal {K})$$ T ∈ B ( H , K ) a linear operator with closed range and Moore–Penrose inverse $$T^\dagger $$ T † . Based on the well-known characterization of proximity operators by Moreau, we prove that for any proximity operator $$\mathrm {Prox}:\mathcal {K}\rightarrow \mathcal {K}$$ Prox : K → K the operator $$T^\dagger \, \mathrm {Prox}( T \cdot + b)$$ T † Prox ( T · + b ) is a proximity operator on $$\mathcal {H}$$ H equipped with a suitable norm. In particular, it follows for the frequently applied soft shrinkage operator $$\mathrm {Prox}= S_{\lambda }:\ell _2 \rightarrow \ell _2$$ Prox = S λ : ℓ 2 → ℓ 2 and any frame analysis operator $$T:\mathcal {H}\rightarrow \ell _2$$ T : H → ℓ 2 that the frame shrinkage operator $$T^\dagger \, S_\lambda \, T$$ T † S λ T is a proximity operator on a suitable Hilbert space. The concatenation of proximity operators on $$\mathbb R^d$$ R d equipped with different norms establishes a PNN. If the network arises from tight frame analysis or synthesis operators, then it forms an averaged operator. In particular, it has Lipschitz constant 1 and belongs to the class of so-called Lipschitz networks, which were recently applied to defend against adversarial attacks. Moreover, due to its averaging property, PNNs can be used within so-called Plug-and-Play algorithms with convergence guarantee. In case of Parseval frames, we call the networks Parseval proximal neural networks (PPNNs). Then, the involved linear operators are in a Stiefel manifold and corresponding minimization methods can be applied for training of such networks. Finally, some proof-of-the concept examples demonstrate the performance of PPNNs.

show abstract

“…As future work, apart from the mathematical analysis of the EM algorithm with approximate M-step, we intend to work on the robustness of the method. This could be done by using a robust PCA [18], and also by making the model invariant to contrast changes, see, e.g. [8].…”

Section: Discussionmentioning

confidence: 99%

PCA Reduced Gaussian Mixture Models with Applications in Superresolution

Hertrich,

Nguyen,

Aujol

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Despite the rapid development of computational hardware, the treatment of large and high dimensional data sets is still a challenging problem. This paper provides a twofold contribution to the topic. First, we propose a Gaussian Mixture Model in conjunction with a reduction of the dimensionality of the data in each component of the model by principal component analysis, called PCA-GMM. To learn the (low dimensional) parameters of the mixture model we propose an EM algorithm whose M-step requires the solution of constrained optimization problems. Fortunately, these constrained problems do not depend on the usually large number of samples and can be solved efficiently by an (inertial) proximal alternating linearized minimization algorithm. Second, we apply our PCA-GMM for the superresolution of 2D and 3D material images based on the approach of Sandeep and Jacob. Numerical results confirm the moderate influence of the dimensionality reduction on the overall superresolution result.

show abstract

On the rotational invariant L1-norm PCA

Cited by 9 publications

References 34 publications

On the Robust PCA and Weiszfeld’s Algorithm

On the Robust PCA and Weiszfeld’s Algorithm

Parseval Proximal Neural Networks

PCA Reduced Gaussian Mixture Models with Applications in Superresolution

Contact Info

Product

Resources

About