Fast and Stable Multivariate Kernel Density Estimation by Fast Sum Updating

Abstract

Kernel density estimation and kernel regression are powerful but computationally expensive techniques: a direct evaluation of kernel density estimates at M evaluation points given N input sample points requires a quadratic O(M N ) operations, which is prohibitive for large scale problems. For this reason, approximate methods such as binning with Fast Fourier Transform or the Fast Gauss Transform have been proposed to speed up kernel density estimation. Among these fast methods, the Fast Sum Updating approach is an attractive alternative, as it is an exact method and its speed is independent of the input sample and the bandwidth. Unfortunately, this method, based on data sorting, has for the most part been limited to the univariate case. In this paper, we revisit the fast sum updating approach and extend it in several ways. Our main contribution is to extend it to the general multivariate case for general input data and rectilinear evaluation grid. Other contributions include its extension to a wider class of kernels, including the triangular, cosine and Silverman kernels, its combination with parsimonious additive multivariate kernels, and its combination with a fast approximate k-nearest-neighbors bandwidth for multivariate datasets. Our numerical tests of multivariate regression and density estimation confirm the speed, accuracy and stability of the method. We hope this paper will renew interest for the fast sum updating approach and help solve largescale practical density estimation and regression problems.

Microstructure-informed probability-driven point-particle model for hydrodynamic forces and torques in particle-laden flows

“…For evaluations given sample points, evaluation of a KDE-estimated PDF by naive kernel summation in (3.19) (which we use for model computations in this work) requires a quadratic operations, which may be computationally prohibitive for the practical implementation of the MPP model. However, this issue can be circumvented to a great extent by using efficient approaches that have been proposed over the past years, including data binning with fast Fourier transform, fast sum updating, fast Gauss transform and the dual-tree method (Langrené & Warin 2019). The computational cost can be reduced from quadratic operations to linear or , resulting in a vast improvement of computational efficiency by orders of magnitude compared with naive kernel summation.…”

Dark matter neutrino scattering in the galactic centre with IceCube

“…When computing KDEs, a kernel function is placed over each event, x i and the kernels across all events are summed to determine the overall PDF at any point, x in observable space. In one-dimension [128]:…”

Stochastic Distance Transform: Theory, Algorithms and Applications

“…There are many KDE kernel functions in common use [32]; a subset of those defined for univariate kernels are listed in Table 1, all of which have a finite support on |z| ≤ 1. There are multiple ways of transforming such univariate kernels into multivariate kernels.…”

“…The listed kernels have a support |z| ≤ 1 and take the value 0 outside of this interval and a suitable norm, a radially symmetric multivariate kernel [32,51] with radius r is given by…”