Weighting vectors for machine learning: numerical harmonic analysis applied to boundary detection

Abstract

Metric space magnitude, an active field of research in algebraic topology, is a scalar quantity that summarizes the effective number of distinct points that live in a general metric space. The weighting vector is a closely-related concept that captures, in a nontrivial way, much of the underlying geometry of the original metric space. Recent work has demonstrated that when the metric space is Euclidean, the weighting vector serves as an effective tool for boundary detection. We recast this result and show the weighting vector may be viewed as a solution to a kernelized SVM. As one consequence, we apply this new insight to the task of outlier detection, and we demonstrate performance that is competitive or exceeds performance of state-of-the-art techniques on benchmark data sets. Under mild assumptions, we show the weighting vector, which has computational cost of matrix inversion, can be efficiently approximated in linear time. We show how nearest neighbor methods can approximate solutions to the minimization problems defined by SVMs.Preprint. Under review.

The magnitude vector of images

“…Following this line of thought, magnitude vectors were introduced as a way to characterise the contribution of each data sample to the overall magnitude, such that the sum of the elements of the magnitude vector amounts to the magnitude. As shown in previous works, the magnitude vectors can detect boundaries of a metric space, with boundary points having a larger contribution to magnitude (Bunch et al, 2021).…”

“…It should be also noted that the definition of the magnitude vector is reminiscent of optimising a support vector machine. This connection has been pointed out for the Euclidean norm by Bunch et al (2021).…”

“…To investigate the effect of different metrics on boundary detection, we consider the behaviour of the weighting vector on the 2-dimensional grid. We closely follow the argument of Bunch et al (2021), extending their results to the product space metric in the case when d i are p norms with p = 1, 2 and…”

“…its well-known properties Folland (1999) and the convolution theorem, we can derive an intuitive understanding of the magnitude vector and the effects of specific metrics. In the case of d(•, •) being the Euclidean ( 2 ) norm, it has been shown in (Bunch et al, 2021) that…”

“…where we generalised the result of Bunch et al (2021) to the unit square. We can interpret equation 4 as the weighting v being constant in the interior of the unit square (∂ 2 i is zero in interior) and different to this constant on the boundary.…”