Considering that when the traditional network structure is connected with the pre-trained language model for sentence embedding representation, it relies on simple word weights to generate sentence vectors, which easily ignores the global and contextual semantic details of sentences, reducing the accuracy of sentence representation. To address these issues, we develop a Multidirectional Attention Interaction Construction-Bert Sentence Representation Framework (MAI-CBert). First, the model uses deep attention to transform the input emotional sentences to reduce misunderstanding caused by the unbalanced distribution of word weights in the sentences; at the same time, it applies horizontal and vertical attention to emotional constructions, which are generated from two different directions. Construction vectors, aim to focus on words with salient features in sentences and establish close dependencies. Second, the dynamic interaction strategy is adopted to realize the interaction between attention in different directions, so that the information flow forms a complementary relationship, resulting in more effective sentence vectors. Notably, the loss function is refactored to improve the representation robustness of the model to avoid catastrophic forgetting. Experimental results on the SemEval-14 and ACL-14 baseline datasets demonstrate that the MAI-CBert sentence representation framework is robust and competitive.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.