Stochastic density ratio estimation and its application to feature selection

Abstract

In this work, we deal with a relatively new statistical tool in machine learning: the estimation of the ratio of two probability densities, or density ratio estimation for short. As a side piece of research that gained its own traction, we also tackle the task of parameter selection in learning algorithms based on kernel methods.

Training Neural Networks for Likelihood/Density Ratio Estimation

“…Since the term C = E 0 [r 2 (X)] does not contain the unknown function u(X), it can be omitted from the cost. The cost in (12) corresponds to r(X) > 0 and it is a special case of Example A.1 with α = 0.…”

“…Of course, there already exists a substantial literature addressing the problem of density ratio estimation [5]- [7] (and references therein). We must also mention the usage of these methods in several applications as covariate shift corrections [8], density ratio estimation with dimensionality reduction [9], change detection [10], approximate likelihood estimation [11], feature selection [12], etc. The focus and main tool in all these publications is density ratio estimation and no effort is made to estimate any transformation of this ratio or any other meaningful statistic that occurs in Detection and Hypothesis testing.…”