Hub Co-occurrence Modeling for Robust High-Dimensional kNN Classification

Tomašev, Nenad; Mladenić, Dunja

doi:10.1007/978-3-642-40991-2_41

Cited by 15 publications

(16 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This is both a geometric consequence of their emergence, as well as a consequence of frequent and severe cluster assumption violation in many real-world data sets. Therefore, hubnessaware methods have been proposed for instance selection [10,66], metric learning [35,36,58,64,70] and classification [67,69,74,76]-in order to reduce the negative impact of detrimental hub points in supervised learning. Hubness has also been shown to affect the performance of learning methods under class imbalance in highdimensional data [68].…”

Section: Application Domains and Other Types Of Hubness-aware Methodsmentioning

confidence: 99%

Hubness-Based Clustering of High-Dimensional Data

Tomašev

Radovanović

Mladenić

et al. 2014

Partitional Clustering Algorithms

Self Cite

View full text Add to dashboard Cite

Hubness has recently been established as a significant property of k-nearest neighbor (k-NN) graphs obtained from high-dimensional data using a distance measure, with traits and effects relevant to the cluster structure of data, as well as clustering algorithms. The hubness property is manifested with increasing (intrinsic) data dimensionality. The distribution of data point in-degrees, i.e. the number of times points appear among the k nearest neighbors of other points in the data, becomes highly skewed. This results in hub points that can have in-degrees multiple orders of magnitude higher than expected. In this chapter we review and refine existing work which explains the mechanisms of the phenomenon, establishes the location of hub points near central regions of clusters in the data, and shows how hubness can negatively affect existing clustering algorithms by virtue of hub points lowering between-cluster distance. Next, we review the newly proposed partitional clustering algorithms, based on K-means, which take advantage of hubness by employing hubs in the process of cluster prototype selection. These "soft" K-means extensions avoid premature convergence to suboptimal stable cluster configurations and are able to reach the global optima more often. The algorithms offer significant improvements over the K-means baseline in scenarios involving high-dimensional and noisy data. The improvements stem from a better placement of hub points into clusters, which helps in increasing the between-cluster distance. Finally, we introduce novel clustering algorithms as "kernelized" versions of the most successful hubness-based methods discussed above, that are able to more effectively handle arbitrarily-shaped clusters.

show abstract

Section: Application Domains and Other Types Of Hubness-aware Methodsmentioning

confidence: 99%

Hubness-Based Clustering of High-Dimensional Data

Tomašev

Radovanović

Mladenić

et al. 2014

Partitional Clustering Algorithms

Self Cite

View full text Add to dashboard Cite

show abstract

“…Points that lie closer to those local cluster centers tend to be closer on average to other points in the cluster, which gives them a higher probability of being included in k-nearest neighbor sets. This is why some hub linkage occurs, where pairs of hub points in proximity of the same cluster center often co-occur as neighbors [37].…”

Section: Data Hubness In Many Dimensionsmentioning

confidence: 99%

“…In classification, hubness-aware k-nearest neighbor voting frameworks have been shown to significantly improve kNN classification accuracy [40][39][36] [37]. Hubness information was also shown to be potentially useful in certain types of support-vector machines (SVM) [15].…”

Section: Introductionmentioning

confidence: 99%

Taming the Empirical Hubness Risk in Many Dimensions

Tomašev¹

2015

Proceedings of the 2015 SIAM International Conference on Data Mining

Self Cite

View full text Add to dashboard Cite

The hubness phenomenon has recently come into focus as an important aspect of the curse of dimensionality that affects many instance-based learning systems. It has to do with the long-tailed distribution of instance relevance within the models, where a small number of hub points dominates the analysis and influences many predictions. High data hubness is therefore often linked to poor system performance. In this paper, we re-examine several hubness-aware metric learning strategies that propose to improve system performance by reducing the expected data hubness. Instead of observing only the expected hubness degrees, our comparisons are aimed at evaluating the shape of the induced hubness degree distribution, in order to better estimate the associated hubness risk. It is revealed that the distribution of hubness degree tends to become highly skewed in many dimensions, so that many samples exhibit substantially higher hubness than expected. We argue that the long-tailed highhubness-degree-variance metrics can be susceptible to highly detrimental high-hubness events, even if they reduce the expected hubness of the data. The experiments indicate significant differences between the compared hubness-aware metric learning approaches and show that simhubs entails the lowest overall hubness risk with increasing dimensionality. This is further shown to improve classifier stability for k-nearest neighbor classification methods.keywords: hubness, curse of dimensionality, metric learning, secondary distances, classification 1 Introduction Instance-based learning in many dimensions is known to be quite challenging [7] due to various adverse effects of the curse of dimensionality [3]. The contrast between relevant and irrelevant points is often reduced due to distance concentration [12][18] [20] and nearest neighbors are considered to be far less meaningful in high-dimensional feature spaces [4][10]. Despite the difficulties, it is still possible to extract useful information from the k-nearest neighbor sets and kNN methods remain popular in many domains, including learning under class imbalance [13] and time series classification [43].

show abstract

“…As nearest neighbor classifiers are attractive both from the theoretical and practical points of view, considerable research was performed to enhance nearest neighbor classification. Some of the most promising recent methods were based on the observation that a few time-series tend to be nearest neighbors of surprisingly large amount of other time-series [62]. We refer to this phenomenon as the presence of hubs or hubness for short, and the classifiers that take this phenomenon into account are called hubness-aware classifiers.…”

Section: Introductionmentioning

confidence: 99%

Classification of Electroencephalograph Data: A Hubness-aware Approach

2016

APH

View full text Add to dashboard Cite

show abstract

Hub Co-occurrence Modeling for Robust High-Dimensional kNN Classification

Cited by 15 publications

References 23 publications

Hubness-Based Clustering of High-Dimensional Data

Hubness-Based Clustering of High-Dimensional Data

Taming the Empirical Hubness Risk in Many Dimensions

Classification of Electroencephalograph Data: A Hubness-aware Approach

Contact Info

Product

Resources

About