The Berkelmans–Pries dependency function: A generic measure of dependence between random variables

Abstract

Measuring and quantifying dependencies between random variables (RVs) can give critical insights into a dataset. Typical questions are: ‘Do underlying relationships exist?’, ‘Are some variables redundant?’, and ‘Is some target variable Y highly or weakly dependent on variable X?’ Interestingly, despite the evident need for a general-purpose measure of dependency between RVs, common practice is that most data analysts use the Pearson correlation coefficient to quantify dependence between RVs, while it is recognized that the correlation coefficient is essentially a measure for linear dependency only. Although many attempts have been made to define more generic dependency measures, there is no consensus yet on a standard, general-purpose dependency function. In fact, several ideal properties of a dependency function have been proposed, but without much argumentation. Motivated by this, we discuss and revise the list of desired properties and propose a new dependency function that meets all these requirements. This general-purpose dependency function provides data analysts with a powerful means to quantify the level of dependence between variables. To this end, we also provide Python code to determine the dependency function for use in practice.

Turing and Van Gogh walk into a bar

“…and the Berkelmans-Pries dependency function [16]. Multiple existing methods already use Shapley values, which has been shown to give many nice properties.…”

“…The property that a feature gets zero FI, when Dep (Y |S ∪ {X}) = Dep (Y |S) for all S ∈ Ω f \ {X} is the same notion as a null player in game theory. Berkelmans et al [16] show that Dep (Y |X) = 0, when Y is independent of X.…”

“…Proof. The BP dependency function is non-increasing under functions of X [16]. This means that for any measurable function g, it…”

“…which is the XOR of X 1 and X 2 . Both X 1 and X 2 do not individually give any new information about the distribution of Y , thus v(X 1 ) = v(X 2 ) = 0 (see properties of the BP dependency function [16]). However, collectively they fully determine Y and thus v(X 1 ∪X 2 ) = 1.…”

“…Consider the dataset consisting of two binary features X ∼ U({0, 1}) and a clone X clone := X, where the target variable Y is defined as Y := X. Note that both X and X clone fully determine Y , thus v(X) = v(X clone ) = 1 (see properties of the BP dependency function [16]). Combining X and X clone also fully determines Y , which leads to:…”