Multiple POS Dependency-Aware Mixture of Experts for Frame Identification

Abstract

Frame identification, which is finding the exact evoked frame for a target word in a given sentence, is a fundamental and crucial prerequisite for frame semantic parsing. It is generally seen as a classification task for target words, whose contextual representations are usually obtained using a neural network like BERT as an encoder, and enriched with a joint learning model or the knowledge of FrameNet. However, the distinction at a fine-grained level, such as the delicate differences in the information of syntax and PropBank roles caused by different parts-of-speech (POS) of target words, is neglected. We propose a Multiple POS Dependency-aware Mixture of Experts(MPDaMoE) network that integrates five types of information, consisting of the syntactic information of target words whose POS are nominal, adjectival, adverbial, or prepositional, and the PropBank role information of target words whose POS are only verbal.To better learn such information, a Mixture of Experts network is employed, in which every expert is a Graph Convolutional Network, to incorporate the different dependency information of target words. Our model outperforms state-of-the-art models in experiments on two benchmark datasets, which shows its effectiveness.

Enhancing Simplified Chinese Poetry Comprehension in LLaMA-7B: A Novel Approach to Mimic Mixture of Experts Effect

“…However, challenges in complexity and computational resource requirements have been identified, where the integration of multiple expert networks demands higher computational power [53,54,55,52]. Another significant observation is the improvement in context-aware processing, particularly in languages with complex grammatical structures [56,57,58]. Lastly, studies suggest that while MoE models offer advancements in accuracy and context sensitivity, integrating these models into existing systems poses practical challenges, including model training and deployment complexities [55,59,60].…”