Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification

Abstract

Short text classification has found rich and critical applications in news and tweet tagging to help users find relevant information. Due to lack of labeled training data in many practical use cases, there is a pressing need for studying semi-supervised short text classification. Most existing studies focus on long texts and achieve unsatisfactory performance on short texts due to the sparsity and limited labeled data. In this paper, we propose a novel heterogeneous graph neural network based method for semi-supervised short text classification, leveraging full advantage of few labeled data and large unlabeled data through information propagation along the graph. In particular, we first present a flexible HIN (heterogeneous information network) framework for modeling the short texts, which can integrate any type of additional information as well as capture their relations to address the semantic sparsity. Then, we propose Heterogeneous Graph ATtention networks (HGAT) to embed the HIN for short text classification based on a dual-level attention mechanism, including node-level and type-level attentions. The attention mechanism can learn the importance of different neighboring nodes as well as the importance of different node (information) types to a current node. Extensive experimental results have demonstrated that our proposed model outperforms state-of-the-art methods across six benchmark datasets significantly.

Heterogeneous Graph Neural Networks for Extractive Document Summarization

“…Tu et al (2019) introduced a heterogeneous graph neural network to encode documents, entities and candidates together for multihop reading comprehension. Linmei et al (2019) focused on semi-supervised short text classification and constructed a topic-entity heterogeneous neural graph.…”

A Hybrid Classification Method via Character Embedding in Chinese Short Text With Few Words

“…Li et al proposed [25] a two-level attention networks to identify the sentiment of short text, both local and long-distance dependent features are captured simultaneously by the attention mechanism, and then the attention-based features are utilized to captures more relevant features. Hu et al proposed [26] a heterogeneous graph neural network for semi-supervised short text classification. In this method, a dual-level attention mechanism which include node-level and type-level attention is utilized to learn the importance of neighbour node and different types to a current node.…”

DRTS Parsing with Structure-Aware Encoding and Decoding

“…GNN has received increasing interests for its strong capability of encoding structural information (Kipf and Welling, 2016;Bastings et al, 2017;. GAT is one representative model, which demonstrates success in a number of NLP tasks (Huang and Carley, 2019;Linmei et al, 2019). In this work, we exploit GAT to represent treestructural information for DRTS parsing.…”

“…We propose to improve DRTS parsing by making use of the above structure information, modeling dependency-based syntax of the input sentences as well as the skeleton structure to enhance the baseline model of Liu et al (2019) using Graph Attention Network (GAT) (Veličković et al, 2018), which has been demonstrated effective for tree/graph encoding (Huang and Carley, 2019;Linmei et al, 2019). In particular, we first derive dependency tree structures for each sentence in a paragraph from the Stanford Parser, and then encode them directly via one GAT module, which are fed as inputs for decoding.…”