Bregman Distances in Inverse Problems and Partial Differential Equations

Abstract

The aim of this paper is to provide an overview of recent development related to Bregman distances outside its native areas of optimization and statistics. We discuss approaches in inverse problems and image processing based on Bregman distances, which have evolved to a standard tool in these fields in the last decade. Moreover, we discuss related issues in the analysis and numerical analysis of nonlinear partial differential equations with a variational structure. For such problems Bregman distances appear to be of similar importance, but are currently used only in a quite hidden fashion. We try to work out explicitely the aspects related to Bregman distances, which also lead to novel mathematical questions and may also stimulate further research in these areas. discussion. Besides missing smoothness of the considered functionals and the fact that problems in imaging, inverse problems and partial differential equations are naturally formulated in infinite-dimensional Banach spaces such as the space of functions of bounded variation or Sobolev spaces, which have only been considered in few instances before, a key point is that the motivation for using Bregman distances in these fields often differs significantly from those in optimization and statistics. In the following we want to provide an overview of such questions and consequent developments, keeping an eye on potential directions and questions for future research. We start with a section including definitions, examples and some general properties of Bregman distances, before we survey aspects of Bregman distances in inverse problems and imaging developed in the last decade. Then we proceed to a discussion of Bregman distances in partial differential equations, which is less explicit and hence the main goal is to highlight hidden use of Bregman distances and make the idea more directly accessible for future research. Finally we conclude with a section on related recent developments. Bregman Distances and their Basic PropertiesWe start with a definition of a Bregman distance. In the remainder of this paper, let X be a Banach space and J : X → R ∪ {+∞} be convex functionals. We first recall the definition of subdifferential respectively subgradients.Definition 2.1. The subdifferential of a convex functional J is defined by(2.1) An element p ∈ ∂J(u) is called subgradient.Having defined a subdifferential we can proceed to the definition of Bregman distances, respectively generalized Bregman distances according to [46] Definition 2.2. The (generalized) Bregman distance related to a convex functional J with subgradient p is defined bywhere p ∈ ∂J(u). The symmetric Bregman distance is defined byNote that in the differentiable case, i.e. ∂J(u) being a singleton, we can omit the special subgradient and writeBy the definition of subgradients the nonnegativity is apparent:We can further characterize vanishing Bregman distances as sharing a subgradient:Proposition 2.4. Let J be convex and p ∈ ∂J(u). Then D p J (v, u) = 0 if and only if p ∈ ∂J(v).Since Bregman distances ...

Parametric Majorization for Data-Driven Energy Minimization Methods

“…holds for all Bregman distances [11]. We consider parametrized energies in several variables, yet we always assume (sub)-gradients, Bregman distances and convex conjugates to be with respect to the first argument x.…”

Diagnosis of Alzheimer's disease with Sobolev gradient‐based optimization and 3D convolutional neural network

“…When it is chosen as φ ( x ) = (1/2)‖ x ‖ 2 , gradient descent‐based optimization is applied with the Bregman divergence Div φ ( x , y ) = (1/2)‖ x − y ‖ 2 by assigning φ (. ) function to the norm …”

“…Bregman divergence has also been used in the proposed optimization since it provides a rich framework to find optimal values. [66][67][68][69][70] In the definition of Bregman divergence, which is Div φ = φ(x) − φ(y) − 〈∇φ(y), x − y〉, where the term φ(.) refers to a distance generation function in mirror descent.…”

“…function to the norm. 69,70 Changing the metric to compute gradients leads to a significant impact on the optimization results. 71 Euclidean gradient is the most standard way and implicitly assumed in many gradient descent methods.…”

Improved algorithm for image TV regularization restoration model based on texture and contrast compensation

“…According to p hi = p k + (f -U hl ) / A, 1/ A can depict the fine degree of details updating in one step iterative process. Particularly, when the parameter A tends to infinity, Algorithm I leads to a lot of researches about inverse scale space [24]. When the parameter A is too small, the value of kmax is small.…”