Cross-Stitch Networks for Multi-task Learning

Misra, Ishan; Shrivastava, Abhinav; Gupta, Abhinav; Hebert, Martial

doi:10.1109/cvpr.2016.433

Cited by 1,154 publications

(822 citation statements)

References 57 publications

Supporting

Mentioning

820

Contrasting

Order By: Relevance

“…In the field of computer vision, some transfer and multi-task learning approaches have also been proposed (Li and Hoiem, 2016;Misra et al, 2016). For example, Misra et al (2016) proposed a multi-task learning model to handle different tasks.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks

Hashimoto¹,

Xiong²,

Tsuruoka³

et al. 2017

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

464

334

View full text Add to dashboard Cite

Transfer and multi-task learning have traditionally focused on either a single source-target pair or very few, similar tasks. Ideally, the linguistic levels of morphology, syntax and semantics would benefit each other by being trained in a single model. We introduce a joint many-task model together with a strategy for successively growing its depth to solve increasingly complex tasks. Higher layers include shortcut connections to lower-level task predictions to reflect linguistic hierarchies. We use a simple regularization term to allow for optimizing all model weights to improve one task's loss without exhibiting catastrophic interference of the other tasks. Our single end-to-end model obtains state-of-the-art or competitive results on five different tasks from tagging, parsing, relatedness, and entailment tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

“…For example, Misra et al (2016) proposed a multi-task learning model to handle different tasks. However, they assume that each data sample has annotations for the different tasks, and do not explicitly consider task hierarchies.…”

Section: Related Workmentioning

confidence: 99%

A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks

Hashimoto¹,

Xiong²,

Tsuruoka³

et al. 2017

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

464

334

View full text Add to dashboard Cite

show abstract

“…On the other hand, each task in soft parameter MTL contains its own model and parameters, and the parameters are encouraged to be similar with some regularization. For example, Misra et al [20] connected two separate networks in a soft parameter sharing way. Then the model leverages a unit called cross-stitch to determine how to combine the knowledge learned in other related tasks as task-specific networks.…”

Section: Multi-task Learningmentioning

confidence: 99%

Leveraging Code Generation to Improve Code Retrieval and Summarization via Dual Learning

Xie

Zhang

et al. 2020

Proceedings of the Web Conference 2020

View full text Add to dashboard Cite

Code summarization generates brief natural language description given a source code snippet, while code retrieval fetches relevant source code given a natural language query. Since both tasks aim to model the association between natural language and programming language, recent studies have combined these two tasks to improve their performance. However, researchers have yet been able to effectively leverage the intrinsic connection between the two tasks as they train these tasks in a separate or pipeline manner, which means their performance can not be well balanced. In this paper, we propose a novel end-to-end model for the two tasks by introducing an additional code generation task. More specifically, we explicitly exploit the probabilistic correlation between code summarization and code generation with dual learning, and utilize the two encoders for code summarization and code generation to train the code retrieval task via multi-task learning. We have carried out extensive experiments on an existing dataset of SQL and Python, and results show that our model can significantly improve the results of the code retrieval task over the-state-of-art models, as well as achieve competitive performance in terms of BLEU score for the code summarization task.

show abstract

“…Multi-task learning in human analysis Multi-task learning [26,44] is widely used in human analysis, knowledge transferring between different tasks can benefit both. In [14], the action detector, object detector and HOI classifier are jointly trained to predict human object relationship accurately.…”

Section: Related Workmentioning

confidence: 99%

TRB: A Novel Triplet Representation for Understanding 2D Human Body

Duan

Lin

Jin

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Human pose and shape are two important components of 2D human body. However, how to efficiently represent both of them in images is still an open question. In this paper, we propose the Triplet Representation for Body (TRB) -a compact 2D human body representation, with skeleton keypoints capturing human pose information and contour keypoints containing human shape information. TRB not only preserves the flexibility of skeleton keypoint representation, but also contains rich pose and human shape information. Therefore, it promises broader application areas, such as human shape editing and conditional image generation. We further introduce the challenging problem of TRB estimation, where joint learning of human pose and shape is required. We construct several large-scale TRB estimation datasets, based on popular 2D pose datasets: LSP, MPII, COCO. To effectively solve TRB estimation, we propose a two-branch network (TRB-net) with three novel techniques, namely X-structure (Xs), Directional Convolution (DC) and Pairwise Mapping (PM), to enforce multi-level message passing for joint feature learning. We evaluate our proposed TRB-net and several leading approaches on our proposed TRB datasets, and demonstrate the superiority of our method through extensive evaluations.

show abstract

Cross-Stitch Networks for Multi-task Learning

Cited by 1,154 publications

References 57 publications

A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks

A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks

Leveraging Code Generation to Improve Code Retrieval and Summarization via Dual Learning

TRB: A Novel Triplet Representation for Understanding 2D Human Body

Contact Info

Product

Resources

About