A Constructive Setting for the Problem of Density Ratio Estimation

Vapnik, Vladimir; Braga, Igor; Izmailov, Rauf

doi:10.1137/1.9781611973440.50

Cited by 8 publications

(6 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The approach proposed in this paper shares resemblance to the recent constructive setting of the density ratio estimate (Vapnik et al 2014). That work minimizes the regularized L 2 -norm between the weighted proposal empirical distribution function and the empirical target distribution function, where the importance weights are defined on a set of basis functions.…”

Section: Introductionmentioning

confidence: 99%

“…That work minimizes the regularized L 2 -norm between the weighted proposal empirical distribution function and the empirical target distribution function, where the importance weights are defined on a set of basis functions. Those importance weights are shown in Vapnik et al (2014) to converge in probability to the RadonNikodym derivative, as the number of proposal and target random samples tend to infinity. Our approach does not use a basis function representation of the importance weights, since we are only interested in evaluating the importance weights at the random sample locations (i.e., we associate one weight with each random sample).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optimal $$L_2$$-norm empirical importance weights for the change of probability measure

2016

View full text Add to dashboard Cite

This work proposes an optimization formulation to determine a set of empirical importance weights to achieve a change of probability measure. The objective is to estimate statistics from a target distribution using random samples generated from a (different) proposal distribution. This work considers the specific case in which the proposal distribution from which the random samples are generated is unknown; that is, we have available the samples but no explicit description of their underlying distribution. In this setting, the Radon-Nikodym theorem provides a valid but indeterminable solution to the task, since the distribution from which the random samples are generated is inaccessible. The proposed approach employs the well-defined and determinable empirical distribution function associated with the available samples. The core idea is to compute importance weights associated with the random samples, such that the distance between the weighted proposal empirical distribution function and the desired target distribution function is minimized. The distance metric selected for this work is the L 2 -norm and the importance weights are constrained to define a probability measure. The resulting optimization problem is shown to be a single linear equality and boxconstrained quadratic program. This problem can be solved efficiently using optimization algorithms that scale well to high dimensions. Under some conditions restricting the class

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Optimal $$L_2$$-norm empirical importance weights for the change of probability measure

2016

View full text Add to dashboard Cite

show abstract

“…In this work, we cover this contribution in Chapters 2 and 3. This contribution also appears in (Vapnik et al, 2014): Braga, I., and Izmailov, R. (2014). A constructive setting for the problem of density ratio estimation.…”

Section: Contributions and Organizationmentioning

confidence: 97%

“…When Y is a categorical variable, the estimation of the mutual information essentially depends on the estimation of a nite number of ratios of probability densities (Sugiyama et al, 2011;Vapnik et al, 2014). Previous work has already attempted to cast MI estimation as density ratio estimation (Suzuki et al, 2009).…”

Section: Mutual Information Estimation and Feature Selectionmentioning

confidence: 99%

“…Any estimation error of the ratios that may happen in this direction will be greatly magnied, causing a large error in MI estimation. Fortunately, there is an equivalent formulation for MI (Vapnik et al, 2014) which provides a more robust estimator by considering the better behaved function z log(z) Figure 4.1. Even though z log(z) goes to ∞ faster than log(z) when z → ∞, the value of z will be bounded from above in MI estimation.…”

Section: Estimators Of Mutual Informationmentioning

confidence: 99%

See 1 more Smart Citation

Stochastic density ratio estimation and its application to feature selection

Braga

View full text Add to dashboard Cite

show abstract

Learning under nonstationarity: covariate shift and class‐balance change

Sugiyama

Yamada

Plessis

2013

WIREs Computational Stats

View full text Add to dashboard Cite

One of the fundamental assumptions behind many supervised machine-learning algorithms is that training and test data follow the same probability distribution. However, this important assumption is often violated in practice, for example, because of an unavoidable sample selection bias or nonstationarity of the environment. Owing to violation of the assumption, standard machine-learning methods suffer a significant estimation bias. In this article, we consider two scenarios of such distribution change-the covariate shift where input distributions differ and class-balance change where class-prior probabilities vary in classification-and review semi-supervised adaptation techniques based on importance weighting. FIGURE 1 | Covariate shift. Input distributions change but the conditional distribution of outputs given inputs does not change. (a) Input densities and importance, (b) Learning target function, training samples, and test samples.466

show abstract

A Constructive Setting for the Problem of Density Ratio Estimation

Cited by 8 publications

References 9 publications

Optimal $$L_2$$-norm empirical importance weights for the change of probability measure

Optimal $$L_2$$-norm empirical importance weights for the change of probability measure

Stochastic density ratio estimation and its application to feature selection

Learning under nonstationarity: covariate shift and class‐balance change

Contact Info

Product

Resources

About