A Model-Agnostic Approach to Mitigate Gradient Interference for Multi-Task Learning

Chai, Heyan; Yin, Zhe; Ding, Ye; Liu, Li; Fang, Binxing; Liao, Qing

doi:10.1109/tcyb.2022.3223377

Cited by 10 publications

(2 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2) We will focus on some entities with plenty of links but have a low place in the hierarchy as mentioned in the last subsection and try to analyse them from the perspective of various relation types. Also, extending our model to more complex Riemannian spaces including spherical space and complex space integrating multi-task learning [50] applications will be a practical idea for many real scenarios.…”

Section: Discussionmentioning

confidence: 99%

Extrapolation over temporal knowledge graph via hyperbolic embedding

Jia

Wang

et al. 2023

CAAI Trans on Intel Tech

Self Cite

View full text Add to dashboard Cite

Predicting potential facts in the future, Temporal Knowledge Graph (TKG) extrapolation remains challenging because of the deep dependence between the temporal association and semantic patterns of facts. Intuitively, facts (events) that happened at different timestamps have different influences on future events, which can be attributed to a hierarchy among not only facts but also relevant entities. Therefore, it is crucial to pay more attention to important entities and events when forecasting the future. However, most existing methods focus on reasoning over temporally evolving facts or mining evolutional patterns from known facts, which may be affected by the diversity and variability of the evolution, and they might fail to attach importance to facts that matter. Hyperbolic geometry was proved to be effective in capturing hierarchical patterns among data, which is considered to be a solution for modelling hierarchical relations among facts. To this end, we propose ReTIN, a novel model integrating real-time influence of historical facts for TKG reasoning based on hyperbolic geometry, which provides low-dimensional embeddings to capture latent hierarchical structures and other rich semantic patterns of the existing TKG. Considering both real-time and global features of TKG boosts the adaptation of ReTIN to the ever-changing dynamics and inherent constraints. Extensive experiments on benchmarks demonstrate the superiority of ReTIN over various baselines. The ablation study further supports the value of exploiting temporal information.

show abstract

Section: Discussionmentioning

confidence: 99%

Extrapolation over temporal knowledge graph via hyperbolic embedding

Jia

Wang

et al. 2023

CAAI Trans on Intel Tech

Self Cite

View full text Add to dashboard Cite

show abstract

“…Motivated by the success of multi-task learning in various tasks [63][64][65][66], we derive the overall training loss of MMCL as below:…”

Section: Self-distillation Layermentioning

confidence: 99%

Towards Spoken Language Understanding via Multi-level Multi-grained Contrastive Learning

Cheng,

Xu,

Zhu

et al. 2023

Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

Spoken language understanding (SLU) is a core task in task-oriented dialogue systems, which aims at understanding the user's current goal through constructing semantic frames. SLU usually consists of two subtasks, including intent detection and slot filling. Although there are some SLU frameworks joint modeling the two subtasks and achieving high performance, most of them still overlook the inherent relationships between intents and slots and fail to achieve mutual guidance between the two subtasks. To solve the problem, we propose a multi-level multi-grained SLU framework MMCL to apply contrastive learning at three levels, including utterance level, slot level, and word level to enable intent and slot to mutually guide each other. For the utterance level, our framework implements coarse granularity contrastive learning and fine granularity contrastive learning simultaneously. Besides, we also apply the self-distillation method to improve the robustness of the model. Experimental results and further analysis demonstrate that our proposed model achieves new state-of-the-art results on two public multi-intent SLU datasets, obtaining a 2.6 overall accuracy improvement on the MixATIS dataset compared to previous best models. CCS CONCEPTS• Computing methodologies → Natural language processing; Discourse, dialogue and pragmatics.

show abstract