Optimization of k nearest neighbor density estimates

Fukunaga, Keinosuke; Hostetler, L.D.

doi:10.1109/tit.1973.1055003

Cited by 170 publications

(72 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1 The set of labeled examples is typically very small compared with the set of unlabeled exam-3 ples. Based on such information, Sinkkonen and Kaski [10] proposed a local metric learning method to improve clus-5 tering and visualization results.…”

Section: Article In Pressmentioning

confidence: 99%

“…Instead of choosing the metric manually, 31 a promising approach is to learn the metric from data automatically. This idea can be dated back to some early work on 33 optimizing the metric for k-nearest neighbor density estimation [1]. Later, optimal local metric [2] and optimal global 35 metric [3] were also developed for nearest neighbor classi-37 fication.…”

Section: Introduction 23mentioning

confidence: 99%

See 1 more Smart Citation

Locally linear metric adaptation with application to semi-supervised clustering and image retrieval

Chang

Yeung

2006

Pattern Recognition

View full text Add to dashboard Cite

7Many computer vision and pattern recognition algorithms are very sensitive to the choice of an appropriate distance metric. Some recent research sought to address a variant of the conventional clustering problem called semi-supervised clustering, which performs 9 clustering in the presence of some background knowledge or supervisory information expressed as pairwise similarity or dissimilarity constraints. However, existing metric learning methods for semi-supervised clustering mostly perform global metric learning through a 11 linear transformation. In this paper, we propose a new metric learning method that performs nonlinear transformation globally but linear transformation locally. In particular, we formulate the learning problem as an optimization problem and present three methods for solving 13 it. Through some toy data sets, we show empirically that our locally linear metric adaptation (LLMA) method can handle some difficult cases that cannot be handled satisfactorily by previous methods. We also demonstrate the effectiveness of our method on some UCI data 15 sets. Besides applying LLMA to semi-supervised clustering, we have also used it to improve the performance of content-based image retrieval systems through metric learning. Experimental results based on two real-world image databases show that LLMA significantly 17 outperforms other methods in boosting the image retrieval performance.

show abstract

Section: Article In Pressmentioning

confidence: 99%

Section: Introduction 23mentioning

confidence: 99%

Locally linear metric adaptation with application to semi-supervised clustering and image retrieval

Chang

Yeung

2006

Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…Alippi and Roveri [2,3] demonstrate how to modify the kNN algorithm for use in the streaming case. First, they demonstrate how to appropriately choose k in a data stream which does not exhibit concept drift based on theoretical results from Fukunga [33]. With this framework, they describe how to update the kNN classifier when no concept drift is detected (add new instances to the knowledge base), and when concept drift is detected (remove obsolete examples from the knowledge base).…”

Section: K-nearest Neighbors Based Methodsmentioning

confidence: 99%

Learning from streaming data with concept drift and imbalance: an overview

2012

View full text Add to dashboard Cite

The primary focus of machine learning has traditionally been on learning from data assumed to be sufficient and representative of the underlying fixed, yet unknown, distribution. Such restrictions on the problem domain paved the way for development of elegant algorithms with theoretically provable performance guarantees. As is often the case, however, real-world problems rarely fit neatly into such restricted models. For instance class distributions are often skewed, resulting in the "class imbalance" problem. Data drawn from non-stationary distributions is also common in real-world applications, resulting in the "concept drift" or "non-stationary learning" problem which is often associated with streaming data scenarios. Recently, these problems have independently experienced increased research attention, however, the combined problem of addressing all of the above mentioned issues has enjoyed relatively little research. If the ultimate goal of intelligent machine learning algorithms is to be able to address a wide spectrum of real-world scenarios, then the need for a general framework for learning from, and adapting to, a non-stationary environment that may introduce imbalanced data can be hardly overstated. In this paper, we first present an overview of each of these challenging areas, followed by a comprehensive review of recent research for developing such a general framework.

show abstract

“…In our first experiment, we theoretically compute the optimal choice of k for a fixed partition with M = 3.5×10 4 and N = 1.5×10 4 . We then show the variation of the theoretical and experimental M.S.E.…”

Section: Optimal Selection Of Free Parametersmentioning

confidence: 99%

Optimized intrinsic dimension estimator using nearest neighbor graphs

Sricharan

Raich

Hero

2010

2010 IEEE International Conference on Acoustics, Speech and Signal Processing

View full text Add to dashboard Cite

We develop an approach to intrinsic dimension estimation based on k-nearest neighbor (kNN) distances. The dimension estimator is derived using a general theory on functionals of kNN density estimates. This enables us to predict the performance of the dimension estimation algorithm. In addition, it allows for optimization of free parameters in the algorithm. We validate our theory through simulations and compare our estimator to previous kNN based dimensionality estimation approaches.

show abstract

Optimization of k nearest neighbor density estimates

Cited by 170 publications

References 7 publications

Locally linear metric adaptation with application to semi-supervised clustering and image retrieval

Locally linear metric adaptation with application to semi-supervised clustering and image retrieval

Learning from streaming data with concept drift and imbalance: an overview

Optimized intrinsic dimension estimator using nearest neighbor graphs

Contact Info

Product

Resources

About