Task Similarity Estimation Through Adversarial Multitask Neural Network

Zhou, Fan; Shui, Changjian; Abbasi, Mahdieh; Robitaille, Louis-Émile; Wang, Boyu; Gagné, Christian

doi:10.1109/tnnls.2020.3028022

“…Multitask Learning: Several multitask learning models have been designed which aim to share knowledge across tasks to improve these tasks' performance (Ruder 2017;Hashimoto et al 2016;Zhang and Yang 2017;Guo et al 2018;Vandenhende et al 2020;Gagné 2019;Zhou et al 2020a). For example, Standley et al (2020) proposed a framework where tasks are grouped and learned by exploiting the cooperative and competitive relationships among the tasks.…”

Section: Related Workmentioning

confidence: 99%

MuMu: Cooperative Multitask Learning-Based Guided Multimodal Fusion

Islam

¹

,

Iqbal

²

2022

AAAI

View full text Add to dashboard Cite

Multimodal sensors (visual, non-visual, and wearable) can provide complementary information to develop robust perception systems for recognizing activities accurately. However, it is challenging to extract robust multimodal representations due to the heterogeneous characteristics of data from multimodal sensors and disparate human activities, especially in the presence of noisy and misaligned sensor data. In this work, we propose a cooperative multitask learning-based guided multimodal fusion approach, MuMu, to extract robust multimodal representations for human activity recognition (HAR). MuMu employs an auxiliary task learning approach to extract features specific to each set of activities with shared characteristics (activity-group). MuMu then utilizes activity-group-specific features to direct our proposed Guided Multimodal Fusion Approach (GM-Fusion) for extracting complementary multimodal representations, designed as the target task. We evaluated MuMu by comparing its performance to state-of-the-art multimodal HAR approaches on three activity datasets. Our extensive experimental results suggest that MuMu outperforms all the evaluated approaches across all three datasets. Additionally, the ablation study suggests that MuMu significantly outperforms the baseline models (p

show abstract

“…Even though significant progress has been achieved, recent successes in machine learning, especially in the deep learning area, usually rely on a large amount of labelled data to obtain a small generalization error. In practice, however, acquiring labelled data could be highly prohibitive, e.g., when classifying multiple objects in an image (Long et al 2017), when analyzing patient data in healthcare data analysis (Wang and Pineau 2015;Zhou et al 2021b), or when modelling users' products preferences (Murugesan and Carbonell 2017). Data hungry has become a long-term problem for deep learning.…”

Section: Introductionmentioning

confidence: 99%

“…It has been shown with benefits to reduce the amount of annotated data per task to reach the desired performance. The crucial idea behind MTL is to extract and leverage the knowledge and information shared across the tasks to improve the overall performances (Wang et al 2019b), which can be achieved by task-invariant feature learning (Maurer, Pontil, and Romera-Paredes 2016;Luo, Tao, and Wen 2017) or task relation learning (Zhang and Yeung 2012;Bingel and Søgaard 2017;Zhou et al 2021b). One major issue with most of the existing feature learning approaches is that they only align the marginal distributions P(x) to extract the shared features without taking advantage of label information of the tasks.…”

Section: Introductionmentioning

confidence: 99%

Multi-task Learning by Leveraging the Semantic Information

Zhou¹,

Chaib-draa²,

Wang³

2021

Preprint

Self Cite

1

0

View full text Add to dashboard Cite

One crucial objective of multi-task learning is to align distributions across tasks so that the information between them can be transferred and shared. However, existing approaches only focused on matching the marginal feature distribution while ignoring the semantic information, which may hinder the learning performance. To address this issue, we propose to leverage the label information in multi-task learning by exploring the semantic conditional relations among tasks. We first theoretically analyze the generalization bound of multi-task learning based on the notion of Jensen-Shannon divergence, which provides new insights into the value of label information in multi-task learning. Our analysis also leads to a concrete algorithm that jointly matches the semantic distribution and controls label distribution divergence. To confirm the effectiveness of the proposed method, we first compare the algorithm with several baselines on some benchmarks and then test the algorithms under label space shift conditions. Empirical results demonstrate that the proposed method could outperform most baselines and achieve state-of-the-art performance, particularly showing the benefits under the label shift conditions.

show abstract

“…Even though significant progress has been achieved, recent successes in machine learning, especially in the deep learning area, usually rely on a large amount of labelled data to obtain a small generalization error. In practice, however, acquiring labelled data could be highly prohibitive, e.g., when classifying multiple objects in an image (Long et al 2017), when analyzing patient data in healthcare data analysis (Wang and Pineau 2015;Zhou et al 2021b), or when modelling users' products preferences (Murugesan and Carbonell 2017). Data hungry has become a long-term problem for deep learning.…”

Section: Introductionmentioning

confidence: 99%

Multi-task Learning by Leveraging the Semantic Information

Zhou

¹

,

Chaib-draa

²

,

Wang

³

2021

AAAI

Self Cite

6

0

View full text Add to dashboard Cite

One crucial objective of multi-task learning is to align distributions across tasks so that the information between them can be transferred and shared. However, existing approaches only focused on matching the marginal feature distribution while ignoring the semantic information, which may hinder the learning performance. To address this issue, we propose to leverage the label information in multi-task learning by exploring the semantic conditional relations among tasks. We first theoretically analyze the generalization bound of multi-task learning based on the notion of Jensen-Shannon divergence, which provides new insights into the value of label information in multi-task learning. Our analysis also leads to a concrete algorithm that jointly matches the semantic distribution and controls label distribution divergence. To confirm the effectiveness of the proposed method, we first compare the algorithm with several baselines on some benchmarks and then test the algorithms under label space shift conditions. Empirical results demonstrate that the proposed method could outperform most baselines and achieve state-of-the-art performance, particularly showing the benefits under the label shift conditions.

show abstract

Task Similarity Estimation Through Adversarial Multitask Neural Network

Cited by 19 publications

References 28 publications

MuMu: Cooperative Multitask Learning-Based Guided Multimodal Fusion

MuMu: Cooperative Multitask Learning-Based Guided Multimodal Fusion

Multi-task Learning by Leveraging the Semantic Information

Multi-task Learning by Leveraging the Semantic Information

Contact Info

Product

Resources

About