A Unified Framework for Manifold Landmarking

Abstract

The success of semi-supervised manifold learning is highly dependent on the quality of the labeled samples. Active manifold learning aims to select and label representative landmarks on a manifold from a given set of samples to improve semi-supervised manifold learning. In this paper, we propose a novel active manifold learning method based on a unified framework of manifold landmarking. In particular, our method combines geometric manifold landmarking methods with algebraic ones. We achieve this by using the Gershgorin circle theorem to construct an upper bound on the learning error that depends on the landmarks and the manifold's alignment matrix in a way that captures both the geometric and algebraic criteria. We then attempt to select landmarks so as to minimize this bound by iteratively deleting the Gershgorin circles corresponding to the selected landmarks. We also analyze the complexity, scalability, and robustness of our method through simulations, and demonstrate its superiority compared to existing methods. Experiments in regression and classification further verify that our method performs better than its competitors.Index Terms-Semi-supervised manifold learning, active learning, manifold landmarking, Gershgorin circle theorem.

Fast Manifold Landmarking Using Extreme Eigen-Pairs

“…All alignment matrices share the property that Φ Φ Φ(i, j) = 0 if samples xi and xj are not neighbors in the K-NN graph. For specific computation of Φ Φ Φ, we refer readers to [3].…”

“…However, Min-Cond still suffers from high complexity when computing log(Φ Φ Φ), i.e., Ulog(Σ)U , where the eigen-decomposition of Φ Φ Φ is Φ Φ Φ = UΣU . Recently, [3] combined algebraic and geometric manifold landmarking methods [4,5,7] into Gershgorin circle landmark selection (GCLS) framework, which was optimized after a series of relaxations for fast computation.…”

“…Semi-supervised manifold learning interpolates informatione.g., features or labels-from discrete observed samples on highdimensional data that actually lie on low-dimensional manifolds [1,2]. One essential problem in semi-supervised manifold learning is how to choose samples on a manifold for labelling (called landmarking), such that the performance of subsequent semi-supervised learning can be optimized, given a landmark budget [3,4].…”

“…However, this method suffers from high complexity due to logarithmic computations of a large matrix. Recently, [3] introduced a fast unified strategy to combine geometric and algebraic methods for landmarking, but it has sub-par performance due to too many relaxations. In summary, though algebraic landmarking can produce accurate objectives, the associated optimizations are typically computation-intensive, or grossly simplified from overrelaxation.…”

On active learning methods for manifold data

“…The intuition is that methods that neglect the manifold structure of the instance data may result in worse performance than AL methods that utilize it. This is a question discussed in the recent AL literature (Cai and He 2012;Chen et al 2010;He 2010;Li et al 2014;Xu et al 2017;Zhou and Sun 2014). The objective of this paper is to systematically review the state-of-the-art active learning methods for manifold data and investigate their performance on synthetic manifold data and real-world applications.…”