Linear Bayes policy for learning in contextual-bandits

Abstract

a b s t r a c tMachine and Statistical Learning techniques are used in almost all online advertisement systems. The problem of discovering which content is more demanded (e.g. receive more clicks) can be modeled as a multi-armed bandit problem. Contextual bandits (i.e., bandits with covariates, side information or associative reinforcement learning) associate, to each specific content, several features that define the ''context'' in which it appears (e.g. user, web page, time, region). This problem can be studied in the stochastic/ statistical setting by means of the conditional probability paradigm using the Bayes' theorem. However, for very large contextual information and/or real-time constraints, the exact calculation of the Bayes' rule is computationally infeasible. In this article, we present a method that is able to handle large contextual information for learning in contextual-bandits problems. This method was tested in the Challenge on Yahoo! dataset at ICML2012's Workshop ''new Challenges for Exploration & Exploitation 3'', obtaining the second place. Its basic exploration policy is deterministic in the sense that for the same input data (as a time-series) the same results are obtained. We address the deterministic exploration vs. exploitation issue, explaining the way in which the proposed method deterministically finds an effective dynamic trade-off based solely in the input-data, in contrast to other methods that use a random number generator.

New Detection Method for Gear Faults Based on Kernel Independent Component Analysis and BP Neural Network

“…The KICA is used to eliminate noise and extract sensitive fault signals, and the BPNN is used for fault pattern recognition. The details behind the KICA and GA-BPNN theories refer to [5] and [8]. The workflow of the proposed new method for gear faults is shown in Fig.…”

A Simple SOM Neural Network Based Fault Detection Model for Fault Diagnosis of Rolling Bearings

“…The PCA is used to reduce the dimensionality of the fault feature vector, and the SOM is applied to the fault pattern recognition. The details behind the PCA and SOM theories refer to [5] and [18]. The workflow diagram block of the proposed diagnosis method for the rolling bearing is shown in Fig.…”