Graph topology enhancement for text classification

Abstract

Despite large-scale pre-trained language models have achieved striking results for text classificaion, recent work has raised concerns about the challenge of shortcut learning. In general, a keyword is regarded as a shortcut if it creates a superficial association with the label, resulting in a false prediction. Conversely, shortcut learning can be mitigated if the model relies on robust causal features that help produce sound predictions. To this end, many studies have explored post-hoc interpretable methods to mine shortcuts and causal features for robustness and generalization. However, most existing methods focus only on single word in a sentence and lack consideration of word-group, leading to wrong causal features. To solve this problem, we propose a new Word-Group mining approach, which captures the causal effect of any keyword combination and orders the combinations that most affect the prediction. Our approach bases on effective post-hoc analysis and beam search, which ensures the mining effect and reduces the complexity. Then, we build a counterfactual augmentation method based on the multiple word-groups, and use an adaptive voting mechanism to learn the influence of different augmentated samples on the prediction results, so as to force the model to pay attention to effective causal features. We demonstrate the effectiveness of the proposed method by several tasks on 8 affective review datasets and 4 toxic language datasets, including cross-domain text classificaion, text attack and gender fairness test.

Boosting Convolutional Neural Networks Using a Bidirectional Fast Gated Recurrent Unit for Text Categorization

“…The TextGCN (Yao et al, 2019) builds a single text graph for a corpus-based on word cooccurrence and document word relations using the Bi-LSTM-CNN method. In the same context, Song et al (2022) constructed two different graphs based on contextual information, called sentence graphs and corpus graphs, respectively. Yang et al (2016) employed a significant comprehensive expression to express semantics accurately.…”

Deep graph clustering network by community prediction-guided augmentation

“…To address the sparsity of graph data, we propose a novel approach that focuses on the topological enhancement [16] of graphs through community detection [1] and link prediction [6]. We utilize the Leiden algorithm [20], a popular method for community detection, to identify communities within the graph.…”