CSVD: clustering and singular value decomposition for approximate similarity search in high-dimensional spaces

Castelli, Vittorio; Thomasian, Alexander; Li, Chung-Sheng

doi:10.1109/tkde.2003.1198398

Cited by 66 publications

(65 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The problem of dimensionality reduction has recently received broad attention in areas such as machine learning, computer vision, and information retrieval (Berry, Dumais, & O'Brie, 1995;Castelli, Thomasian, & Li, 2003;Deerwester et al, 1990;Dhillon & Modha, 2001;Kleinberg & Tomkins, 1999;Srebro & Jaakkola, 2003). The goal of dimensionality reduction is to obtain more compact representations of the data with limited loss of information.…”

Section: Introductionmentioning

confidence: 99%

“…The representation of data by vectors in Euclidean space allows one to compute the similarity between data points, based J. YE on the Euclidean distance or some other similarity metric. The similarity metrics on data points naturally lead to similarity-based indexing by representing queries as vectors and searching for their nearest neighbors (Aggarwal, 2001;Castelli, Thomasian, & Li, 2003).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generalized Low Rank Approximations of Matrices

2005

Mach Learn

382

268

View full text Add to dashboard Cite

Abstract. The problem of computing low rank approximations of matrices is considered. The novel aspect of our approach is that the low rank approximations are on a collection of matrices. We formulate this as an optimization problem, which aims to minimize the reconstruction (approximation) error. To the best of our knowledge, the optimization problem proposed in this paper does not admit a closed form solution. We thus derive an iterative algorithm, namely GLRAM, which stands for the Generalized Low Rank Approximations of Matrices. GLRAM reduces the reconstruction error sequentially, and the resulting approximation is thus improved during successive iterations. Experimental results show that the algorithm converges rapidly.We have conducted extensive experiments on image data to evaluate the effectiveness of the proposed algorithm and compare the computed low rank approximations with those obtained from traditional Singular Value Decomposition (SVD) based methods. The comparison is based on the reconstruction error, misclassification error rate, and computation time. Results show that GLRAM is competitive with SVD for classification, while it has a much lower computation cost. However, GLRAM results in a larger reconstruction error than SVD. To further reduce the reconstruction error, we study the combination of GLRAM and SVD, namely GLRAM + SVD, where SVD is preceded by GLRAM. Results show that when using the same number of reduced dimensions, GLRAM + SVD achieves significant reduction of the reconstruction error as compared to GLRAM, while keeping the computation cost low.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Generalized Low Rank Approximations of Matrices

2005

Mach Learn

382

268

View full text Add to dashboard Cite

show abstract

“…5 Ω, where p is a small oversampling parameter (typically set to [5][6][7][8][9][10]. Multiplying A with the random matrix Ω, we obtain Y = AΩ.…”

Section: Preliminariesmentioning

confidence: 99%

“…From each cluster a concept vector is derived which is later used as a basis to obtain a low rank approximation of A. In [9,41], similarly to [14], clustering is used to partition either the rows or columns of an m × n matrix A. An approximation of the data matrix A is obtained using rank-1 or rank-k i truncated SVD approximations of each row or column cluster.…”

Section: Principal Angles Assume We Have a Truncated Svd Approximatimentioning

confidence: 99%

See 1 more Smart Citation

Clustered Matrix Approximation

Savas¹,

Dhillon²

2016

SIAM J. Matrix Anal. & Appl.

View full text Add to dashboard Cite

Abstract. In this paper we develop a novel clustered matrix approximation framework, first showing the motivation behind our research. The proposed methods are particularly well suited for problems with large scale sparse matrices that represent graphs and/or bipartite graphs from information science applications. Our framework and resulting approximations have a number of benefits: (1) the approximations preserve important structure that is present in the original matrix; (2) the approximations contain both global-scale and local-scale information; (3) the procedure is efficient both in computational speed and memory usage; and (4) the resulting approximations are considerably more accurate with less memory usage than truncated SVD approximations, which are optimal with respect to rank. The framework is also quite flexible as it may be modified in various ways to fit the needs of a particular application. In the paper we also derive a probabilistic approach that uses randomness to compute a clustered matrix approximation within the developed framework. We further prove deterministic and probabilistic bounds of the resulting approximation error. Finally, in a series of experiments we evaluate, analyze, and discuss various aspects of the proposed framework. In particular, all the benefits we claim for the clustered matrix approximation are clearly illustrated using real-world and large scale data.

show abstract

An Incremental Updating Method for Clustering-Based High-Dimensional Data Indexing

Wang

Gan

2005

Computational Intelligence and Security

View full text Add to dashboard Cite

CSVD: clustering and singular value decomposition for approximate similarity search in high-dimensional spaces

Cited by 66 publications

References 25 publications

Generalized Low Rank Approximations of Matrices

Generalized Low Rank Approximations of Matrices

Clustered Matrix Approximation

An Incremental Updating Method for Clustering-Based High-Dimensional Data Indexing

Contact Info

Product

Resources

About