The image labeling problem can be described as assigning to each pixel a single element from a finite set of predefined labels. Recently, a smooth geometric approach for inferring such label assignments was proposed by following the Riemannian gradient flow of a given objective function on the so-called assignment manifold. Due to the specific Riemannian structure, this results in a coupled replicator dynamic incorporating local spatial geometric averages of lifted data-dependent distances. However, in this framework an approximation of the flow is necessary in order to arrive at explicit formulas. We discuss preliminary results of an alternative model where lifting and averaging is decoupled in the objective function so as to stay closer to established approaches and at the same time preserve the ingredients of the original approach. As a consequence the resulting flow is explicitly given, without the need for any approximation, while still exploiting the underlying Riemannian structure.
Motivation and PreliminariesSuppose f : V → F is a given image on a set of pixels V = {1, . . . , m} with values in some feature space F which might be a manifold. A labeling on V given f assigns to every pixel i ∈ V a label l(i) ∈ L from a predefined set of labels L = {l 1 , . . . , l n }. In [1] a new smooth geometric approach for image labeling was proposed. The basic idea is to encode (relaxed) labelings as points on the assignment manifold given by the set of row-stochastic matrices with full support, which is turned into a Riemannian manifold by choosing the Fisher-Rao (information) metric. In this geometric setting, every row W i := (W i1 , . . . , W in ) of an assignment matrix W ∈ W represents a probability assignment of labels L at pixel i ∈ V. Furthermore, a distance function d : F × L → R measuring the fit of labels to the data is assumed to be given. All the distance information is collected into a global matrix D := (d(f i , l j )) i,j ∈ R m×n and lifted onto W depending on the current assignment W . The Riemannian mean of these lifted distances in a spatial neighborhood around every pixel i ∈ V is used to define the similarity matrix S ∈ W. A labeling is inferred by maximizing the correlation between the current assignment W and the spatial geometric averages S(W, D) following the Riemannian gradient ascent flow on W. After ignoring the slow dynamics in the model and approximating the Riemannian mean to first order by the geometric mean, the resulting dynamical model in [1] is given by spatially coupled replicator equations called assignment flow.While this new approach results in fast numerical updates with good performance, there are some open mathematical aspects. It can be shown that the resulting assignment flow is not variational anymore, i.e. there exists no potential in the Fisher-Rao geometry. This is a consequence of the mentioned simplifying assumption and approximation, which are unavoidable for achieving explicit formulas and efficient numerical schemes. In the following we briefly present an alternat...
