Multi-Task Boosting by Exploiting Task Relationships

Zhang, Yu; Yeung, Dit–Yan

doi:10.1007/978-3-642-33460-3_50

Cited by 36 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been proved in [4], [5] that problem ( 21) is jointly convex with respect to W, b, and Ω. Problem (21) has been extended to multi-task boosting [83] and multi-label learning [84] by learning label correlations. Problem (21) can also been interpreted from the perspective of reproducing kernel Hilbert spaces for vector-valued functions [85], [86], [87], [88].…”

Section: Task Relation Learning Approachmentioning

confidence: 99%

A Survey on Multi-Task Learning

Zhang

Yang

2017

Preprint

Self Cite

309

298

View full text Add to dashboard Cite

Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL. First, we classify different MTL algorithms into several categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach, and decomposition approach, and then discuss the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, batch MTL models are difficult to handle this situation and online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing are reviewed to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works. Finally, we present theoretical analyses and discuss several future directions for MTL.

show abstract

Section: Task Relation Learning Approachmentioning

confidence: 99%

A Survey on Multi-Task Learning

Zhang

Yang

2017

Preprint

Self Cite

309

298

View full text Add to dashboard Cite

show abstract

“…One way is to group tasks into several clusters in which the tasks in the same group are regarded as related [Bakker and Heskes 2003;Jacob et al 2008;Romera-Paredes et al 2012;Zhou et al 2011]. The other way is to learn the intertask covariance matrix of the multivariate Gaussian prior which can model both positive and negative task correlations [Fei and Huan 2013;Saha et al 2011;Zhang and Yeung 2010, 2012a, 2012b, 2014. Zhang [2015] proposed the convex DMTRC algorithm based on the second way.…”

Section: Negative Transfermentioning

confidence: 98%

Smart Multitask Bregman Clustering and Multitask Kernel Clustering

Zhang

Liu

2015

ACM Trans. Knowl. Discov. Data

View full text Add to dashboard Cite

Traditional clustering algorithms deal with a single clustering task on a single dataset. However, there are many related tasks in the real world, which motivates multitask clustering. Recently some multitask clustering algorithms have been proposed, and among them multitask Bregman clustering (MBC) is a very applicable method. MBC alternatively updates clusters and learns relationships between clusters of different tasks, and the two phases boost each other. However, the boosting does not always have positive effects on improving the clustering performance, it may also cause negative effects. Another issue of MBC is that it cannot deal with nonlinear separable data. In this article, we show that in MBC, the process of using cluster relationship to boost the cluster updating phase may cause negative effects, that is, cluster centroids may be skewed under some conditions. We propose a smart multitask Bregman clustering (S-MBC) algorithm which can identify the negative effects of the boosting and avoid the negative effects if they occur. We then propose a multitask kernel clustering (MKC) framework for nonlinear separable data by using a similar framework like MBC in the kernel space. We also propose a specific optimization method, which is quite different from that of MBC, to implement the MKC framework. Since MKC can also cause negative effects like MBC, we further extend the framework of MKC to a smart multitask kernel clustering (S-MKC) framework in a similar way that S-MBC is extended from MBC. We conduct experiments on 10 real world multitask clustering datasets to evaluate the performance of S-MBC and S-MKC. The results on clustering accuracy show that: (1) compared with the original MBC algorithm MBC, S-MBC and S-MKC perform much better; (2) compared with the convex discriminative multitask relationship clustering (DMTRC) algorithms DMTRC-L and DMTRC-R which also avoid negative transfer, S-MBC and S-MKC perform worse in the (ideal) case in which different tasks have the same cluster number and the empirical label marginal distribution in each task distributes evenly, but better or comparable in other (more general) cases. Moreover, S-MBC and S-MKC can work on the datasets in which different tasks have different number of clusters, violating the assumptions of DMTRC-L and DMTRC-R. The results on efficiency show that S-MBC and S-MKC consume more computational time than MBC and less computational time than DMTRC-L and DMTRC-R. Overall S-MBC and S-MKC are competitive compared with the state-of-the-art multitask clustering algorithms in synthetical terms of accuracy, efficiency and applicability.

show abstract

“…Multi-Task Learning. State-of-the-art algorithms on multitask learning [46], [20], [47], [48], [49], [25], [50] can also be roughly divided into two categories: (1) Regularized multi-task feature learning methods [46], [20], [48], which assume all tasks are homogeneous and the learning is to discover common feature representation across all tasks. (2) Task relationship exploration methods [47], [49], [25], [51], which either exploit task relationships via trace norm regularization to achieve some similar parameters among similar tasks [51], or try to learn a task covariance matrix from data if the task relationship is not known in advance [47], [25].…”

Section: Related Workmentioning

confidence: 99%

Task Sensitive Feature Exploration and Learning for Multitask Graph Classification

Pan

Zhu

et al. 2017

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Multitask learning (MTL) is commonly used for jointly optimizing multiple learning tasks. To date, all existing MTL methods have been designed for tasks with feature-vector represented instances, but cannot be applied to structure data, such as graphs. More importantly, when carrying out MTL, existing methods mainly focus on exploring overall commonality or disparity between tasks for learning, but cannot explicitly capture task relationships in the feature space, so they are unable to answer important questions, such as what exactly is shared between tasks and what is the uniqueness of one task differing from others? In this paper, we formulate a new multitask graph learning problem, and propose a task sensitive feature exploration and learning algorithm for multitask graph classification. Because graphs do not have features available, we advocate a task sensitive feature exploration and learning paradigm to jointly discover discriminative subgraph features across different tasks. In addition, a feature learning process is carried out to categorize each subgraph feature into one of three categories: (1) common feature; (2) task auxiliary feature; and (3) task specific feature, indicating whether the feature is shared by all tasks, by a subset of tasks, or by only one specific task, respectively. The feature learning and the multiple task learning are iteratively optimized to form a multitask graph classification model with a global optimization goal. Experiments on real-world functional brain analysis and chemical compound categorization demonstrate the algorithm's performance. Results confirm that our method can be used to explicitly capture task correlations and uniqueness in the feature space, and explicitly answer what are shared between tasks and what is the uniqueness of a specific task.

show abstract

Multi-Task Boosting by Exploiting Task Relationships

Cited by 36 publications

References 17 publications

A Survey on Multi-Task Learning

A Survey on Multi-Task Learning

Smart Multitask Bregman Clustering and Multitask Kernel Clustering

Task Sensitive Feature Exploration and Learning for Multitask Graph Classification

Contact Info

Product

Resources

About