A Survey on Multi-Task Learning

Zhang, Yu; Yang, Qiang

doi:10.48550/arxiv.1707.08114

Cited by 309 publications

(320 citation statements)

References 159 publications

(171 reference statements)

Supporting

Mentioning

319

Contrasting

Order By: Relevance

“…Multi-task Learning. Multi-task learning [30][38] [38][38] is a machine learning paradigm that aims to leverage useful information across multiple tasks to improve performance of each task, which has many applications in various areas such as natural language processing [32], computer vision [15][3] and speech recognition [4] [29]. In early literature of multi-task learning, [5] proposes a kernel method and regularization model to learn multi-task simultaneously.…”

Section: Related Workmentioning

confidence: 99%

Leaving No One Behind: A Multi-Scenario Multi-Task Meta Learning Approach for Advertiser Modeling

Zhang

Liao

Liu

et al. 2022

Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

View full text Add to dashboard Cite

Advertisers play an essential role in many e-commerce platforms like Taobao and Amazon. Fulfilling their marketing needs and supporting their business growth is critical to the long-term prosperity of platform economies. However, compared with extensive studies on user modeling such as click-through rate predictions, much less attention has been drawn to advertisers, especially in terms of understanding their diverse demands and performance. Different from user modeling, advertiser modeling generally involves many kinds of tasks (e.g. predictions of advertisers' expenditure, active-rate, or total impressions of promoted products). In addition, major e-commerce platforms often provide multiple marketing scenarios (e.g. Sponsored Search, Display Ads, Live Streaming Ads) while advertisers' behavior tend to be dispersed among many of them. This raises the necessity of multi-task and multi-scenario consideration in comprehensive advertiser modeling, which faces the following challenges: First, one model per scenario or per task simply doesn't scale; Second, it is particularly hard to model new or minor scenarios with limited data samples; Third, inter-scenario correlations are complicated, and may vary given different tasks.To tackle these challenges, we propose a multi-scenario multitask meta learning approach (M2M) which simultaneously predicts multiple tasks in multiple advertising scenarios. Specifically, we introduce a novel meta unit that incorporates rich scenario knowledge to learn explicit inter-scenario correlations and can easily scale to new scenarios. Furthermore, we present a meta attention module to capture diverse inter-scenario correlations given different tasks, and a meta tower module to enhance the capability of capturing the representation of scenario-specific features. Compelling results from both offline evaluation and online A/B tests demonstrate the superiority of M2M over state-of-the-art methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Leaving No One Behind: A Multi-Scenario Multi-Task Meta Learning Approach for Advertiser Modeling

Zhang

Liao

Liu

et al. 2022

Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

View full text Add to dashboard Cite

show abstract

“…Multi-Objective Optimization (MOO) refers to the paradigm of learning multiple related objectives together [5,7,31,58]. Recently, there are four thrusts of MOO methods that consider the negative transfer problem.…”

Section: Related Workmentioning

confidence: 99%

“…However, the commonly-encountered datasparsity problem often hurts the generalization of the learned matching model [15,34], which is usually due to lacking annotation of matching relationships. To tackle this problem, Multi-source Entity-Matching (MEM) which exploits data from one or several auxiliary sources acts as a mainstream solution [6,8,25,58,60,61]. Nevertheless, existing MEM approaches usually assume that data distributions/spaces are shared between sources, or sufficient annotations of entity correspondence between sources can be acquired [6,8,25,60,61], which may not hold in real-applications.…”

Section: Introductionmentioning

confidence: 99%

Incentive Compatible Pareto Alignment for Multi-Source Large Graphs

Liang¹,

Lv²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we focus on learning effective entity matching models over multi-source large-scale data. For real applications, we relax typical assumptions that data distributions/spaces, or entity identities are shared between sources, and propose a Relaxed Multi-source Large-scale Entity-matching (RMLE) problem. Challenges of the problem include 1) how to align large-scale entities between sources to share information and 2) how to mitigate negative transfer from joint learning multi-source data. What's worse, one practical issue is the entanglement between both challenges. Specifically, incorrect alignments may increase negative transfer; while mitigating negative transfer for one source may result in poorly learned representations for other sources and then decrease alignment accuracy. To handle the entangled challenges, we point out that the key is to optimize information sharing first based on Pareto front optimization, by showing that information sharing significantly influences the Pareto front which depicts lower bounds of negative transfer. Consequently, we proposed an Incentive Compatible Pareto Alignment (ICPA) method to first optimize cross-source alignments based on Pareto front optimization, then mitigate negative transfer constrained on the optimized alignments. This mechanism renders each source can learn based on its true preference without worrying about deteriorating representations of other sources. Specifically, the Pareto front optimization encourages minimizing lower bounds of negative transfer, which optimizes whether and which to align. In detail, we adopt graph neural networks to handle data sparsity in each source and a scalable alignment based on sliced graph matching. Comprehensive empirical evaluation results on four large-scale datasets are provided to demonstrate the effectiveness and superiority of ICPA. Online A/B test results at a search advertising platform also demonstrate the effectiveness of ICPA in production environments. We also release an International Entity Graph (IEG) 1 dataset to facilitate future research.

show abstract

“…It aims to first learn a global model and then efficiently adapt it to individual robots while minimizing the extra training cost of personalization. This technique is inspired by multi-task learning [22], [27] and meta-learning [26], and is especially important in situations where the cost of personalization is relatively high, such as for low-power robots or mobile phones. Another approach to personalized FL is to weigh the parameters of a global model and unique personalized local model.…”

Section: Related Workmentioning

confidence: 99%

Personalized Federated Learning of Driver Prediction Models for Autonomous Driving

Nakanoya¹,

Im²,

Qiu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Autonomous vehicles (AVs) must interact with a diverse set of human drivers in heterogeneous geographic areas. Ideally, fleets of AVs should share trajectory data to continually re-train and improve trajectory forecasting models from collective experience using cloud-based distributed learning. At the same time, these robots should ideally avoid uploading raw driver interaction data in order to protect proprietary policies (when sharing insights with other companies) or protect driver privacy from insurance companies. Federated learning (FL) is a popular mechanism to learn models in cloud servers from diverse users without divulging private local data. However, FL is often not robust -it learns sub-optimal models when user data comes from highly heterogeneous distributions, which is a key hallmark of human-robot interactions. In this paper, we present a novel variant of personalized FL to specialize robust robot learning models to diverse user distributions. Our algorithm outperforms standard FL benchmarks by up to 2× in real user studies that we conducted where human-operated vehicles must gracefully merge lanes with simulated AVs in the standard CARLA and CARLO AV simulators.

show abstract

A Survey on Multi-Task Learning

Cited by 309 publications

References 159 publications

Leaving No One Behind: A Multi-Scenario Multi-Task Meta Learning Approach for Advertiser Modeling

Leaving No One Behind: A Multi-Scenario Multi-Task Meta Learning Approach for Advertiser Modeling

Incentive Compatible Pareto Alignment for Multi-Source Large Graphs

Personalized Federated Learning of Driver Prediction Models for Autonomous Driving

Contact Info

Product

Resources

About