Information Geometry of U-Boost and Bregman Divergence

Motomura, Noboru; Takenouchi, Takashi; Kanamori, Takafumi; Eguchi, Shinto

doi:10.1162/089976604323057452

Cited by 149 publications

(123 citation statements)

References 15 publications

Supporting

Mentioning

121

Contrasting

Unclassified

Order By: Relevance

“…For properties and applications of Bregman geometry see Eguchi (2005), Murata et al (2004). However most of these do not depend on the Bregman form, and continue to hold for a general decision geometry.…”

Section: Bregman Geometrymentioning

confidence: 99%

The geometry of proper scoring rules

Dawid

2006

AISM

125

View full text Add to dashboard Cite

A decision problem is defined in terms of an outcome space, an action space and a loss function. Starting from these simple ingredients, we can construct: Proper Scoring Rule; Entropy Function; Divergence Function; Riemannian Metric; and Unbiased Estimating Equation. From an abstract viewpoint, the loss function defines a duality between the outcome and action spaces, while the correspondence between a distribution and its Bayes act induces a self-duality. Together these determine a "decision geometry" for the family of distributions on outcome space. This allows generalisation of many standard statistical concepts and properties. In particular we define and study generalised exponential families. Several examples are analysed, including a general Bregman geometry.

show abstract

Section: Bregman Geometrymentioning

confidence: 99%

The geometry of proper scoring rules

Dawid

2006

AISM

125

View full text Add to dashboard Cite

show abstract

“…The first and typical one in machine learning community is AdaBoost [19] for the minimization of the exponential loss. Other boosting methods for various objective function such as likelihood, L 2 -loss, mixture of the exponential loss and naive loss, U-loss, AUC and pAUC [4], [5], [7], [13], [14], [20] have been considered and applied to real data analysis. However, the boosting methods for other purpose than prediction seem to have been paid little attention, see [21]- [23].…”

Section: Boosting For Density Estimationmentioning

confidence: 99%

“…Boosting satisfies a great applicability for minimization of various loss functions. A class of U-loss functions is discussed with a close association with U-entropy and Udivergence [5], [6], where U is a generator function on the real line such as an exponential function. Any U-loss function can employ the idea of boosting with a simple change from AdaBoost.…”

Section: Introductionmentioning

confidence: 99%

Boosting Learning Algorithm for Pattern Recognition and Beyond

Komori

Eguchi

2011

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

SUMMARYThis paper discusses recent developments for pattern recognition focusing on boosting approach in machine learning. The statistical properties such as Bayes risk consistency for several loss functions are discussed in a probabilistic framework. There are a number of loss functions proposed for different purposes and targets. A unified derivation is given by a generator function U which naturally defines entropy, divergence and loss function. The class of U-loss functions associates with the boosting learning algorithms for the loss minimization, which includes AdaBoost and LogitBoost as a twin generated from Kullback-Leibler divergence, and the (partial) area under the ROC curve. We expand boosting to unsupervised learning, typically density estimation employing U-loss function. Finally, a future perspective in machine learning is discussed.

show abstract

“…In particular, this extension is useful for proposing boosting methods [10][11][12][13][14][15][16].…”

Section: Power Divergencementioning

confidence: 99%

Entropy and Divergence Associated with Power Function and the Statistical Application

Eguchi

Kato

2010

Entropy

Self Cite

View full text Add to dashboard Cite

In statistical physics, Boltzmann-Shannon entropy provides good understanding for the equilibrium states of a number of phenomena. In statistics, the entropy corresponds to the maximum likelihood method, in which Kullback-Leibler divergence connects Boltzmann-Shannon entropy and the expected log-likelihood function. The maximum likelihood estimation has been supported for the optimal performance, which is known to be easily broken down in the presence of a small degree of model uncertainty. To deal with this problem, a new statistical method, closely related to Tsallis entropy, is proposed and shown to be robust for outliers, and we discuss a local learning property associated with the method.

show abstract

Information Geometry of U-Boost and Bregman Divergence

Cited by 149 publications

References 15 publications

The geometry of proper scoring rules

The geometry of proper scoring rules

Boosting Learning Algorithm for Pattern Recognition and Beyond

Entropy and Divergence Associated with Power Function and the Statistical Application

Contact Info

Product

Resources

About