The energy distance for ensemble and scenario reduction

Abstract

Scenario reduction techniques are widely applied for solving sophisticated dynamic and stochastic programs, especially in energy and power systems, but are also used in probabilistic forecasting, clustering and estimating generative adversarial networks. We propose a new method for ensemble and scenario reduction based on the energy distance which is a special case of the maximum mean discrepancy. We discuss the choice of energy distance in detail, especially in comparison to the popular Wasserstein distance which is dominating the scenario reduction literature. The energy distance is a metric between probability measures that allows for powerful tests for equality of arbitrary multivariate distributions or independence. Thanks to the latter, it is a suitable candidate for ensemble and scenario reduction problems. The theoretical properties and considered examples indicate clearly that the reduced scenario sets tend to exhibit better statistical properties for the energy distance than a corresponding reduction with respect to the Wasserstein distance. We show applications to a Bernoulli random walk and two real data-based examples for electricity demand profiles and day-ahead electricity prices. This article is part of the theme issue ‘The mathematics of energy systems’.

Collective proposal distributions for nonlinear MCMC samplers: Mean-field theory and fast implementation

“…where X, X ∼ μ and Y, Y ∼ ν are independent. In the machine learning literature, it is often understood as a kernel norm comparable to other popular distances such as the Wasserstein distance [36,87]. This interpretation comes from the formulation…”

Introduction: the mathematics of energy systems

“…The question of optimality is linked to the choice of a suitable distance measure. While the Wasserstein distance has been widely used in this context, the article by Ziel [5] proposes the use of the so-called energy distance, a special case of the maximum mean discrepancy, for ensemble and scenario reduction. This approach is supported by the finding that the energy distance can be used for testing for equality of arbitrary multivariate distributions or for their independence.…”