Minimum distance histograms with universal performance guarantees

Abstract

We present a data-adaptive multivariate histogram estimator of an unknown density f based on n independent samples from it. Such histograms are based on binary trees called regular pavings (RPs). RPs represent a computationally convenient class of simple functions that remain closed under addition and scalar multiplication. Unlike other density estimation methods, including various regularization and Bayesian methods based on the likelihood, the minimum distance estimate (MDE) is guaranteed to be within an L 1 distance bound from f for a given n, no matter what the underlying f happens to be, and is thus said to have universal performance guarantees (Devroye and Lugosi, Combinatorial methods in density estimation. Springer, New York, 2001). Using a form of tree matrix arithmetic with RPs, we obtain the first generic constructions of an MDE, prove that it has universal performance guarantees and demonstrate its performance with simulated and real-world data. Our main contribution is a constructive implementation of an MDE histogram that can handle large multivariate data bursts using a tree-based partition that is computationally conducive to subsequent statistical operations.

Scalable Multivariate Histograms

“…The c Ξ many initial conditions from the core supportcarved path for launching the SEB-PQMCs, should be well spread out in order to facilitate a better search strategy over S 0:m , and in particular, contain the SEB-PQMC launched from the root node as one of its joint SPC/SEB-PQMC paths. In essence, we can view the joint exploration as one that will necessarily improve upon the estimate found by the SEB-PQMC initialized from the root node (which satisfies the three conditions of [8] and thus asymptotically L 1consistent as shown in [10]).…”

Special feature: information theory and statistics

“…The author also shows that the minimum decoding error probability decreases with an exponential order that increases linearly with block length. - Sainudiin and Teng (2019) present a data-adaptive multivariate histogram estimator of an unknown density based on independent samples. The authors prove the universal performance guarantee under the L 1 distance.…”