Dual Principal Component Pursuit

Abstract

We consider the problem of learning a linear subspace from data corrupted by outliers. Classical approaches are typically designed for the case in which the subspace dimension is small relative to the ambient dimension. Our approach works with a dual representation of the subspace and hence aims to find its orthogonal complement; as such, it is particularly suitable for subspaces whose dimension is close to the ambient dimension (subspaces of high relative dimension). We pose the problem of computing normal vectors to the inlier subspace as a non-convex 1 minimization problem on the sphere, which we call Dual Principal Component Pursuit (DPCP) problem. We provide theoretical guarantees under which every global solution to DPCP is a vector in the orthogonal complement of the inlier subspace. Moreover, we relax the non-convex DPCP problem to a recursion of linear programs whose solutions are shown to converge in a finite number of steps to a vector orthogonal to the subspace. In particular, when the inlier subspace is a hyperplane, the solutions to the recursion of linear programs converge to the global minimum of the non-convex DPCP problem in a finite number of steps. We also propose algorithms based on alternating minimization and iteratively re-weighted least squares, which are suitable for dealing with large-scale data. Experiments on synthetic data show that the proposed methods are able to handle more outliers and higher relative dimensions than current state-of-the-art methods, while experiments in the context of the three-view geometry problem in computer vision suggest that the proposed methods can be a useful or even superior alternative to traditional RANSAC-based approaches for computer vision and other applications.

Provable Self-Representation Based Outlier Detection in a Union of Subspaces

“…Baselines. We compare with 6 other representative methods that are designed for detecting outliers in one or multiple subspaces: CoP [40], OutlierPursuit [55], REAPER [21], DPCP [46], LRR [28] and 1 -thresholding [43]. We also compare with a graph based method: OutRank [34,35].…”

Coherence Pursuit: Fast, Simple, and Robust Principal Component Analysis

“…Similarly, the solver of [43] has O(nm 2 + m 3 ) complexity per iteration. In [31], the complement of the column space of L is recovered via a series of linear optimization problems, each obtaining one direction in the complement space. This method is sensitive to structured outliers, particularly linearly dependent outliers, and requires the columns of L not to exhibit a clustering structure, which prevails much of the real world data.…”

“…This method is sensitive to structured outliers, particularly linearly dependent outliers, and requires the columns of L not to exhibit a clustering structure, which prevails much of the real world data. Also, the approach presented in [31] requires solving m − r linear optimization problems consecutively resulting in high computational complexity and long run time for high-dimensional data.…”

An Overview of Robust Subspace Recovery

“…In these tests, we do not compare with DPCP [103], since the code provided online is really just an iterative application of a slower version of the FMS algorithm, and generally DPCP is meant for the setting of large d.…”