Kernel-based Outlier Detection using the Inverse Christoffel Function

Abstract

Outlier detection methods have become increasingly relevant in recent years due to increased security concerns and because of its vast application to different fields. Recently, Pauwels and Lasserre (2016) noticed that the sublevel sets of the inverse Christoffel function accurately depict the shape of a cloud of data using a sumof-squares polynomial and can be used to perform outlier detection. In this work, we propose a kernelized variant of the inverse Christoffel function that makes it computationally tractable for data sets with a large number of features. We compare our approach to current methods on 15 different data sets and achieve the best average area under the precision recall curve (AUPRC) score, the best average rank and the lowest root mean square deviation.

Relating Leverage Scores and Density using Regularized Christoffel Functions

“…Our main result quantitatively describes a decreasing relation between leverage score and population density for a broad class of kernels on Euclidean spaces. Numerical simulations support our findings.1 Kernelized Christoffel functions were first proposed by Laurent El Ghaoui and independently studied in [4].32nd Conference on Neural Information Processing Systems (NIPS 2018),…”

“…1 Kernelized Christoffel functions were first proposed by Laurent El Ghaoui and independently studied in [4].…”

Data-Driven Reachability analysis and Support set Estimation with Christoffel Functions

“…The present paper significantly extends the theory of finite-sample error bounds for support set estimators derived from Christoffel functions by applying techniques from Bayesian PAC analysis, a variation of classical PAC analysis that has been successfully applied to Gaussian process classifiers [29], kernel support vector machines [18], and minimum-volume covering ellipsoids [12]. This extension leverages a formal connection between the kernel empirical inverse Christoffel function investigated by Askari et al [2] and the posterior variance of a Gaussian process regression model. In conjunction with the PAC-Bayes theorem, the connection can be used to derive finite-sample bounds for kernelized empirical inverse Christoffel functions.…”

“…When the measure in question is defined by a probability distribution on R n the level sets of Christoffel functions are known empirically to provide tight approximations to the support. This support-approximating quality has motivated the use of Christoffel functions in several statistical applications, such as density estimation [19,20] and outlier detection [2]. Additionally, the level sets have been shown, using the plug-in approach [6], to converge exactly to the support of the distribution (in the sense of Hausdorff measure) when the degree of the polynomial approaches infinity and when the true probability distribution is available [20].…”

“…The support of a random variable is the range of values it can assume: for example, if X admits a probability density function p X , then the support of X is the closure of the set {x : p X (x) = 0}. In addition to the control-theoretic application developed above, support set estimation has several applications in statistics and data science, such as outlier and novelty detection [26,25,2]. It is therefore useful to consider the problem for general random variables: we will do so for the theoretical developments in this paper, returning to the reachability application in the numerical examples of Section 4.…”