The paper introduces scaled Bregman distances of probability distributions which admit non-uniform contributions of observed events. They are introduced in a general form covering not only the distances of discrete and continuous stochastic observations, but also the distances of random processes and signals. It is shown that the scaled Bregman distances extend not only the classical ones studied in the previous literature, but also the information divergence and the related wider class of convex divergences of probability measures. An information processing theorem is established too, but only in the sense of invariance w.r.t. statistically sufficient transformations and not in the sense of universal monotonicity. Pathological situations where coding can increase the classical Bregman distance are illustrated by a concrete example. In addition to the classical areas of application of the Bregman distances and convex divergences such as recognition, classification, learning and evaluation of proximity of various features and signals, the paper mentions a new application in 3D-exploratory data analysis. Explicit expressions for the scaled Bregman distances are obtained in general exponential families, with concrete applications in the binomial, Poisson and Rayleigh families, and in the families of exponential processes such as the Poisson and diffusion processes including the classical examples of the Wiener process and geometric Brownian motion.

No abstract

No abstract

No abstract

We introduce Bregman distances between general probability measures, and discuss some link with Csiszar divergences. From this, we derive bounds on particular Bregman distances between non-lognormally distributed financial diffusion processes. General considerationsLet P H resp. P A be a (say, hypothesis resp. alternative) probability measure on a measurable space (Ω, F ), both dominated by a σ-finite measure λ on (Ω, F ) with Radon-Nikodym densities f H = dP H /dλ, f A = dP A /dλ. Furthermore, (in adaptation to λ) let φ : ]0, ∞[ → ] − ∞, ∞[ be a sufficiently smooth, strictly convex function (continuously extended into the form, where here and for the rest of Section 1 we always assume that the involved integrals (as well as sums and derivatives) exist. In the spirit of [1], let us introduce the "measure-theoretic" (MT) Bregman distancewhere g φ (f H , · ; λ) formally denotes the corresponding directional derivative at f H . For the special case φ(ρ) := ρ log ρ, the MT Bregman distance d φ (P A , P H ; λ) reduces to the the Kullback-Leibler information divergence (relative entropy) which -as any more general Csiszar φ-divergence -does not depend on λ, see e.g. [3]. However, in general the MT Bregman distance d φ does depend on the choice of the dominating measure λ. To see this, let us take Ω = IR, P H to be any probability measure on IR, P A to be a probability measure on IR with density h w.r.t. P H satisfying P H [h = 1] < 1, as well as φ(ρ) := (ρ − 1)2 . Then with λ 1 := P H and λ 2 := 2P H one gets d φ (P A , P H ; λ 1 ) = d(h, 1; P H ) = I R (h − 1) 2 dP H as wellWith this non-arbitrariness in mind, for the case Ω ⊂ IR one might fix a prominent reference measure such as the Lebesgue measure (see e.g. [6] for the case φ(ρ) := ρ log ρ), or λ dis := i∈I N xi with Dirac measure xi sitting on point x i ∈ IR. Accordingly, for any two discrete probability measures P dis H := i∈I N p [5]. For the case P A P H (on general Ω) and φ(1) = 0, another particular reference measure choice is λ := P H which leads toPut into words, this means that in such a situation the concepts of MT Bregman distance d φ and Csiszar φ-divergence D φ coincide. In contrast, for the abovementioned discrete setup the connection between d φ (P Application to financial diffusion processesLet X t be the value of a financial asset at time t, and suppose that its dynamics can be modeled by either (H) or (A): (H) a generally non-lognormally distributed diffusion process X t which is the (unique, weak) solution of the SDE (2) is denoted by P H; (0,x) . (A) a generally non-lognormally distributed diffusion process X t which is the (unique, weak) solution of the SDE dX t = g A (t, X t ) X t dt + σ(t) X t dW t , 0 ≤ t ≤ T < ∞,