Optimized residual vector quantization for efficient approximate nearest neighbor search

Ai, Liefu; Yu, Jiguo; Wu, Zebin; He, Yansong; Guan, Tao

doi:10.1007/s00530-015-0470-9

Cited by 22 publications

(28 citation statements)

References 33 publications

(47 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, there are scalar quantization algorithms specifically designed to optimize evaluation metrics of the fields [20,21]. Furthermore, for high-dimensional vector quantization, there have been multiple codebook-based algorithms with various optimization strategies [22,23]. However, despite these developments in the area, to the best of the authors' knowledge, hitherto there has not been publication discussing scalar quantization algorithms as sparse least square optimization.…”

Section: Related Workmentioning

confidence: 99%

Scalar Quantization as Sparse Least Square Optimization

Wang

Yang

Fei

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Quantization can be used to form new vectors/matrices with shared values close to the original. In recent years, the popularity of scalar quantization for value-sharing application has been soaring as it has been found huge utilities in reducing the complexity of neural networks. Existing clusteringbased quantization techniques, while being well-developed, have multiple drawbacks including the dependency of the random seed, empty or out-of-the-range clusters, and high time complexity for large number of clusters. To overcome these problems, in this paper, the problem of scalar quantization is examined from a new perspective, namely sparse least square optimization. Specifically, inspired by the property of sparse least square regression, several quantization algorithms based on l 1 least square are proposed. In addition, similar schemes with l 1 + l 2 and l 0 regularization are proposed. Furthermore, to compute quantization results with given amount of values/clusters, this paper designed an iterative method and a clustering-based method, and both of them are built on sparse least square. The paper shows that the latter method is mathematically equivalent to an improved version of k-means clustering-based quantization algorithm, although the two algorithms originated from different intuitions. The algorithms proposed were tested with three types of data and their computational performances, including information loss, time consumption, and the distribution of the values of the sparse vectors, were compared and analyzed. The paper offers a new perspective to probe the area of quantization, and the algorithms proposed can outperform existing methods especially under some bit-width reduction scenarios, when the required post-quantization resolution (number of values) is not significantly lower than the original number.

show abstract

Section: Related Workmentioning

confidence: 99%

Scalar Quantization as Sparse Least Square Optimization

Wang

Yang

Fei

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…However, the contribution of each layer to the minimization of the quantization error is not the same, as the last layers contribute much less than the first ones. This algorithm has been extended by several other methods [17], [18].…”

Section: Residual Vector Quantizationmentioning

confidence: 99%

“…The authors introduce several layers of quantization, as each layer quantizes the residuals of the previous layer. Improvements on this method have been proposed in [17], [18]. In [19], Babenko et al propose a method again based on quantization by addition of several codevectors, yet they remove the hierarchy between layers of residual quantization, providing a high level of freedom for the quantization codevectors.…”

mentioning

confidence: 99%

Competitive Quantization for Approximate Nearest Neighbor Search

Ozan

Kıranyaz

Gabbouj

2016

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

In this study, we propose a novel vector quantization algorithm for Approximate Nearest Neighbor (ANN) search, based on a joint competitive learning strategy and hence called as competitive quantization (CompQ). CompQ is a hierarchical algorithm, which iteratively minimizes the quantization error by jointly optimizing the codebooks in each layer, using a gradient decent approach. An extensive set of experimental results and comparative evaluations show that CompQ outperforms the-state-of-the-art while retaining a comparable computational complexity.

show abstract

“…Data clustering is a fundamental data analysis tool in the area of data mining [9,10], pattern recognition [11,12,41], image analysis [47,48], feature extraction [13,14], vector quantization [15,16], image segmentation [17,18], function approximation [19,20], dimensionality reduction [49,50] and big data analysis [21,22]. It is a process of revealing inherent structures present in a set of objects based on proximity measure and competitive learning.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Functional Link Artificial Neural Networks for Cluster Analysis

et al. 2020

View full text Add to dashboard Cite

In this paper, we propose a novel method of cluster analysis called unsupervised functional link artificial neural networks (UFLANNs), which inherit the best characteristics of functional link artificial neural networks and self-organizing feature maps (SOFMs). UFLANNs adopt three types of basis functions such as Chebyshev, Legendre orthogonal polynomials, and power series for mapping the input data into a new feature space with higher dimensions, where the objects are clustered based on the principle of competitive learning of SOFMs. The effectiveness of this algorithm has been tested with various artificial and real-life datasets including remote sensing images. A thorough comparison with other popular clustering algorithms shows that the proposed method is promising in revealing clusters from many complex datasets.

show abstract

Optimized residual vector quantization for efficient approximate nearest neighbor search

Cited by 22 publications

References 33 publications

Scalar Quantization as Sparse Least Square Optimization

Scalar Quantization as Sparse Least Square Optimization

Competitive Quantization for Approximate Nearest Neighbor Search

Unsupervised Functional Link Artificial Neural Networks for Cluster Analysis

Contact Info

Product

Resources

About