The fractional Kullback–Leibler divergence

Abstract

The Kullback–Leibler divergence or relative entropy is generalised by deriving its fractional form. The conventional Kullback–Leibler divergence as well as other formulations emerge as special cases. It is shown that the fractional divergence encapsulates different relative entropy states via the manipulation of the fractional order and for this reason it is the evolution equation for relative entropy. The fractional Kullback–Leibler divergence establishes mathematical dualities with other divergences or distance metrics. The fractional-order can be characterised as a distance metric between divergences or relative entropy states. Generalised asymptotic divergences and densities are derived that are mixtures of known approaches.

Dualities and Asymptotic Mixtures Using Functional-Order Differentiation

“…The first case where α = α 0 has always been dealt with in the literature. The second case α = α(y) has recently been discussed in [6]. The case of α = α(x) will be examined here.…”

“…, M and M providing the total number of parameters for each density. It has been shown that the fractional entropy for probability densities is given by [6]:…”