An External Knowledge Enhanced Graph-based Neural Network for Sentence Ordering

Abstract

As an important text coherence modeling task, sentence ordering aims to coherently organize a given set of unordered sentences. To achieve this goal, the most important step is to effectively capture and exploit global dependencies among these sentences. In this paper, we propose a novel and flexible external knowledge enhanced graph-based neural network for sentence ordering. Specifically, we first represent the input sentences as a graph, where various kinds of relations (i.e., entity-entity, sentence-sentence and entity-sentence) are exploited to make the graph representation more expressive and less noisy. Then, we introduce graph recurrent network to learn semantic representations of the sentences. To demonstrate the effectiveness of our model, we conduct experiments on several benchmark datasets. The experimental results and in-depth analysis show our model significantly outperforms the existing state-of-the-art models.

Improving Graph-based Sentence Ordering with Iteratively Predicted Pairwise Orderings

“…Recently, inspired by the great success of deep learning in other NLP tasks, researchers have resorted to neural sentence ordering models, which can be classified into: pairwise ordering models Agrawal et al, 2016;Li and Jurafsky, 2017;Moon et al, 2019;Kumar et al, 2020;Prabhumoye et al, 2020;Zhu et al, 2021) and set-to-sequence models (Gong et al, 2016;Nguyen and Joty, 2017;Logeswaran et al, 2018;Mohiuddin et al, 2018;Cui et al, 2018;Yin et al, 2019;Oh et al, 2019;Yin et al, 2020;Cui et al, 2020;Yin et al, 2021). Generally, the former predicts the relative orderings between pairwise sentences, which are then leveraged to produce the final ordered sentence sequence.…”

“…In this paper, we propose a novel iterative pairwise ordering prediction framework which introduces two classifiers to make better use of pairwise orderings for graph-based sentence ordering (Yin et al, 2019(Yin et al, , 2021. As an extension of Sentence-Enity Graph Recurrent Network (SE-GRN) (Yin et al, 2019(Yin et al, , 2021, our framework enriches the graph representation with iteratively predicted orderings between pairwise sentences, which further benefits the subsequent generation of ordered sentences. The basic intuitions behind our work are two-fold.…”

“…Concretely, we propose two graph-based classifiers to iteratively conduct ordering predictions for pairwise sentences. The first classifier takes the sentence-entity graph (SE-Graph) (Yin et al, 2019(Yin et al, , 2021 as input and yields relative orderings of linked sentences via corresponding probabilities. Next, in an iterative manner, the second classifier enriches the previous graph representation by converting high-value probabilities into the weights of the corresponding edges, and then reconduct graph encoding to predict orderings for the other pairwise sentences.…”

“…However, there is little attempt to combine these two types of models, which inituitively possess complementary advantages. In this paper, we propose a novel sentence ordering framework which introduces two classifiers to make better use of pairwise orderings for graph-based sentence ordering (Yin et al, 2019(Yin et al, , 2021. Specially, given an initial sentence-entity graph, we first introduce a graph-based classifier to predict pairwise orderings between linked sentences.…”

“…Then, in an iterative manner, based on the graph updated by previously predicted highconfident pairwise orderings, another classifier is used to predict the remaining uncertain pairwise orderings. At last, we adapt a GRN-based sentence ordering model (Yin et al, 2019(Yin et al, , 2021 on the basis of final graph. Experiments on five commonly-used datasets demonstrate the effectiveness and generality of our model.…”