Subspace Least Squares Multidimensional Scaling

Abstract

Multidimensional Scaling (MDS) is one of the most popular methods for dimensionality reduction and visualization of high dimensional data. Apart from these tasks, it also found applications in the field of geometry processing for the analysis and reconstruction of non-rigid shapes. In this regard, MDS can be thought of as a \textit{shape from metric} algorithm, consisting of finding a configuration of points in the Euclidean space that realize, as isometrically as possible, some given distance structure. In the present work we cast the least squares variant of MDS (LS-MDS) in the spectral domain. This uncovers a multiresolution property of distance scaling which speeds up the optimization by a significant amount, while producing comparable, and sometimes even better, embeddings.Comment: Scale Space and Variational Methods in Computer Vision: 6th International Conference, SSVM 2017, Kolding, Denmark, June 4-8, 201

Multidimensional Scaling

“…Subspace approaches for distance scaling (i.e., stress based) restrict the embedding coordinates X, or rather the displacement field δ ≡ X − X 0 , to some linear subspace Φ ∈ R n×p , i.e., δ = Φα. In [17], the SMACOF iteration (13) was reformulated to the case where δ is modeled as a p-bandlimited signal on some manifold M 0 , i.e., it can be approximated by a linear combination of the first p eigenvectors of the Laplace-Beltrami operator. This is justified by the fact that since we are looking for a distance preserving embedding, δ should be smooth in the sense that close points on M 0 should remain close on the final embedding.…”

“…The matrix D contains geodesic distances computed on the homer shape between 5103 points. The algorithms compared are the original SMACOF algorithm [10], an RRE vector extrapolation acceleration method [19], and spectral SMACOF [17]. By embedding only 200 points and extrapolating to the rest using the Laplace-Beltrami eigenbasis (the green line), we achieve significant speedup compared to the other methods.…”

“…To cope with these problems, a robust MDS model was proposed by [28]. Note that the following transformation applied to a PSD matrix K produces a squared-Euclidean distance matrix (see also (17)):…”

“…While the number of iterations until convergence drops dramatically, the cost per iteration becomes prohibitive for even moderate values of n, making second-order methods impractical. Several works [15][16][17] propose to split the MDS problem into several subproblems ("resolutions" or "scales") by embedding only a subset of carefully chosen points. The coarse embedding obtained from this sample set is then interpolated and refined.…”

“…Specially constructed decimation and interpolation operators allow moving between scales, but since these are ad hoc constructions, they are hard to generalize for different manifolds. A spectral approach was described in [17] which achieves better performance and generalizes easily across manifolds.…”