A theory of transfer learning with applications to active learning

Abstract. We study the optimal rates of convergence for estimating a prior distribution over a VC class from a sequence of independent data sets respectively labeled by independent target functions sampled from the prior. We specifically derive upper and lower bounds on the optimal rates under a smoothness condition on the correct prior, with the number of samples per data set equal the VC dimension. These results have implications for the improvements achievable via transfer learning. We additionally extend this setting to real-valued function, where we establish consistency of an estimator for the prior, and discuss an additional application to a preference elicitation problem in algorithmic economics.

show abstract

“…The proof of this result parallels that of [12] for the transfer learning setting, but is included here for completeness.…”

Section: Maximizing Customer Satisfaction In Combinatorial Auctionssupporting

confidence: 53%

“…The proof parallels that of [12] (who studied the special case of binary functions), with a few important twists (in particular, a significantly different approach in the analogue of their Lemma 3). The details are included in Appendix A.…”

Section: Consistent Estimation Of Priors Over Real-valued Functions Amentioning

confidence: 61%

Bounds on the Minimax Rate for Estimating a Prior over a VC Class from Independent Learning Tasks

Yang

Hanneke

Carbonell

2015

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Another line of related research is transfer learning (Do & Ng, 2005;Yang, Hanneke, & Carbonell, 2013;Zhang, Ghahramani, & Yang, 2008). We seek to transfer prerequisite relations between pairs of courses within universities to other pairs also within universities, and to pairs that span universities.…”

Section: Discussion and Related Workmentioning

confidence: 99%

Learning Concept Graphs from Online Educational Data

Liu¹,

Ma²,

Yang³

et al. 2016

jair

Self Cite

View full text Add to dashboard Cite

This paper addresses an open challenge in educational data mining, i.e., the problem of automatically mapping online courses from different providers (universities, MOOCs, etc.) onto a universal space of concepts, and predicting latent prerequisite dependencies (directed links) among both concepts and courses. We propose a novel approach for inference within and across course-level and concept-level directed graphs. In the training phase, our system projects partially observed course-level prerequisite links onto directed concept-level links; in the testing phase, the induced concept-level links are used to infer the unknown courselevel prerequisite links. Whereas courses may be specific to one institution, concepts are shared across different providers. The bi-directional mappings enable our system to perform interlingua-style transfer learning, e.g. treating the concept graph as the interlingua and transferring the prerequisite relations across universities via the interlingua. Experiments on our newly collected datasets of courses from MIT, Caltech, Princeton and CMU show promising results.

show abstract

“…Another line of related research is transfer learning [7,25,26]. We seek to transfer prerequisite relations between pairs of courses within universities to other pairs also within universities, and to pairs that span universities.…”

Section: Discussion and Related Workmentioning

confidence: 99%

Concept Graph Learning from Educational Data

Yang

Liu

Carbonell

et al. 2015

Proceedings of the Eighth ACM International Conference on Web Search and Data Mining

Self Cite

View full text Add to dashboard Cite

This paper addresses an open challenge in educational data mining, i.e., the problem of using observed prerequisite relations among courses to learn a directed universal concept graph, and using the induced graph to predict unobserved prerequisite relations among a broader range of courses. This is particularly useful to induce prerequisite relations among courses from different providers (universities, MOOCs, etc.). We propose a new framework for inference within and across two graphs-at the course level and at the induced concept level-which we call Concept Graph Learning (CGL). In the training phase, our system projects the course-level links onto the concept space to induce directed concept links; in the testing phase, the concept links are used to predict (unobserved) prerequisite links for test-set courses within the same institution or across institutions. The dual mappings enable our system to perform an interlingua-style transfer learning, e.g. treating the concept graph as the interlingua, and inducing prerequisite links in a transferable manner across different universities. Experiments on our newly collected data sets of courses from MIT, Caltech, Princeton and CMU show promising results, including the viability of CGL for transfer learning.

show abstract

A theory of transfer learning with applications to active learning

Cited by 144 publications

References 19 publications

Bounds on the Minimax Rate for Estimating a Prior over a VC Class from Independent Learning Tasks

Bounds on the Minimax Rate for Estimating a Prior over a VC Class from Independent Learning Tasks

Learning Concept Graphs from Online Educational Data

Concept Graph Learning from Educational Data

Contact Info

Product

Resources

About