Transductive Rademacher Complexity and Its Applications

Abstract

We develop a technique for deriving data-dependent error bounds for transductive learning algorithms based on transductive Rademacher complexity. Our technique is based on a novel general error bound for transduction in terms of transductive Rademacher complexity, together with a novel bounding technique for Rademacher averages for particular algorithms, in terms of their "unlabeled-labeled" representation. This technique is relevant to many advanced graph-based transductive algorithms and we demonstrate its effectiveness by deriving error bounds to three well known algorithms. Finally, we present a new PAC-Bayesian bound for mixtures of transductive algorithms based on our Rademacher bounds.

Towards Active Learning on Graphs: An Error Bound Minimization Approach

“…In this subsection, we derive a generalization error bound for LLGC using the tool of transductive Rademacher complexity for general function classes [8].…”

“…In particular, we choose Learning with Local and Global Consistency (LLGC) [17] as the classifier on graphs, because it is comparable to or even better than Minimum Cut (MinCut) [4] and GFHF [18]. We present a data-dependent generalization error bound for LLGC using the tool of transductive Rademacher Complexity [8], which is an extension of inductive Rademacher Complexity [2] and measures the richness of a class of real-valued functions with respect to a probability distribution. We show that the empirical transductive Rademacher complexity is a good surrogate for active learning on graphs.…”

Generalization Error Bounds for Kernel Matrix Completion and Extrapolation

“…In this section, we derive bounds for the GE of the MC in (1), KMC in (2) and KKMCEX in (4) algorithms. There are two approaches to GE analysis, namely the inductive [24] and the transductive one in [25]. In the inductive one GE measures the difference between the expected value of a loss function and the empirical loss over a finite number of samples.…”

“…However, this definition of GE does not fit the MC framework because it assumes that: i) the data distribution is known; and, ii) the entries are sampled with repetition. In order to come up with distributionfree claims for MC, one may resort to the transductive GE analysis [25]. In this scenario, we are given S n = S m ∪ S u of n data comprising the union of the training set S m and the testing set S u , where |S u | = u.…”

“…Definition 1. Transductive Rademacher complexity [25] Given a set S n = S m ∪ S u with q := 1 u + 1 m , the transductive Rademacher complexity (TRC) of a matrix class F is…”