Boosted multi-task learning

Abstract

In this paper we propose a novel algorithm for multi-task learning with boosted decision trees. We learn several different learning tasks with a joint model, explicitly addressing their commonalities through shared parameters and their differences with taskspecific ones. This enables implicit data sharing and regularization. Our algorithm is derived using the relationship between 1 -regularization and boosting. We evaluate our learning method on web-search ranking data sets from several countries. Here, multi-task learning is particularly helpful as data sets from different countries vary largely in size because of the cost of editorial judgments. Further, the proposed method obtains state-of-the-art results on a publicly available multi-task dataset. Our experiments validate that learning various tasks jointly can lead to significant improvements in performance with surprising reliability.

Domain Adaptation for Microscopy Imaging

“…We will show that our approach consistently outperforms recent multi-task learning techniques [8], [18], [11] across this wide range of applications. Our approach was first introduced in a conference paper [19].…”

“…Current approaches to Domain Adaptation, and more generally Transfer or Multi-Task Learning [5], [6], [7], [8], treat classification in each domain as separate but related problems and exploit their relationship to learn from the supervised data available across all of them. Multi-task learning methods typically assume that the decision boundaries in each domain can be decomposed into a private and a shared term in a common feature space X , as illustrated by Fig.…”

“…In this paper we develop a boosting-based approach [9], [10], [8] that can simultaneously learn the non-linear mappings as well as the shared decision boundary. We boost regression trees or stumps and model the domain-specific mappings with a set of common regression trees that are shared across domains, but whose thresholds have been adapted to each of them.…”

“…More recent multi-task learning methods jointly optimize over both the shared and task-specific components of each task [7], [24], [8], [25]. In [7] it was shown how the two step iterative optimization of [6] can be cast into a single convex optimization problem.…”

“…Feature-based methods generally require well established cross-domain correspondences and/or model non-linearities using the kernel-trick that relies on the selection of a predefined kernel function and is difficult to scale to large datasets. In this paper, we pursue a discriminative learning approach that does not require explicit cross-domain correspondences, and exploits the boosting-trick [8], [9] to handle non-linearities and learn a shared representation across tasks, overcoming these limitations.…”