Dependency parsing masking and BERT for sentiment analysis

Abstract

Sentiment Analysis (SA) is one of the most challenging tasks in natural language processing. It is the process of analyzing, processing, generalizing, and reasoning about subjective text with emotional overtones. To cope with sentiment analysis tasks, current deep learning approaches typically use large-scale pre-trained language models and attention mechanisms that apply computed attention weights without any restrictions on attention allocation. This mechanism is not an ideal model for dealing with the task of sentiment analysis, because in all kinds of language used in life, a semantically complete sentence is composed of words with different semantics connected to a complex and diverse grammar. Factors influencing the sentiment tendency of a sentence include the meaning of words and the change in semantics by the superposition of grammars with different structures. This paper proposes a sentiment analysis model that incorporates information about dependency syntax structure. Dependent syntactic information treats words as nodes and establishes syntactic grammatical structures between words by establishing node-to-node relationships. The undirected graph composed of the dependent syntactic structure information screens the text embedding derived from the attention mechanism in multiple layers, so that each layer focuses only on the associated text, and the sentiment classifier uses the text information containing the syntactic structure of the grammar in the classification. Experiments on the Online_Shopping_10_Cats dataset show a significant improvement in the final results compared to current state-of-the-art sentiment analysis models.