ESSR: Evolving Sparse Sharing Representation for Multitask Learning

Zhang, Yayu; Qian, Yuhua; Ma, Guoshuai; Liang, Xinyan; Liu, Guoqing; Zhang, Qingfu; Tang, Ke

doi:10.1109/tevc.2023.3272663

Cited by 3 publications

(1 citation statement)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These randomized approaches at times yield unstable, uncertain, and unexplainable experimental outcomes. The ESSR method (Zhang et al 2023) acquires task sparse representations through evolutionary processes. However, its feasibility is restricted for extensive computer vision tasks due to the demand for substantial computational resources.…”

Section: Introductionmentioning

confidence: 99%

Learning Multi-Task Sparse Representation Based on Fisher Information

Zhang,

Qian,

et al. 2024

AAAI

View full text Add to dashboard Cite

Multi-task learning deals with multiple related tasks simultaneously by sharing knowledge. In a typical deep multi-task learning model, all tasks use the same feature space and share the latent knowledge. If the tasks are weakly correlated or some features are negatively correlated, sharing all knowledge often leads to negative knowledge transfer among. To overcome this issue, this paper proposes a Fisher sparse multi-task learning method. It can obtain a sparse sharing representation for each task. In such a way, tasks share features on a sparse subspace. Our method can ensure that the knowledge transferred among tasks is beneficial. Specifically, we first propose a sparse deep multi-task learning model, and then introduce Fisher sparse module into traditional deep multi-task learning to learn the sparse variables of task. By alternately updating the neural network parameters and sparse variables, a sparse sharing representation can be learned for each task. In addition, in order to reduce the computational overhead, an heuristic method is used to estimate the Fisher information of neural network parameters. Experimental results show that, comparing with other methods, our proposed method can improve the performance for all tasks, and has high sparsity in multi-task learning.

show abstract

Section: Introductionmentioning

confidence: 99%