Deep Attentive Sentence Ordering Network

Abstract

In this paper, we propose a novel deep attentive sentence ordering network (referred as ATTOrderNet) which integrates self-attention mechanism with LSTMs in the encoding of input sentences. It enables us to capture global dependencies among sentences regardless of their input order and obtains a reliable representation of the sentence set. With this representation, a pointer network is exploited to generate an ordered sequence. The proposed model is evaluated on Sentence Ordering and Order Discrimination tasks. The extensive experimental results demonstrate its effectiveness and superiority to the state-ofthe-art methods.

Topic-Guided Coherence Modeling for Sentence Ordering by Preserving Global and Local Information

“…Table 3: Performance of predicting the correct head and tail sentences on arXiv and SIND. The results are directly taken from (Cui et al, 2018).…”

“…This section reports experimental results on the sentence ordering task for determining a coherent order of a given sentence. The proposed TGCM was compared with state-of-theart methods as baselines such as Pairwise Model , Seq2seq (Logeswaran et al, 2018), RNN Decoder (Logeswaran et al, 2018), V-LSTM+PtrNet (Logeswaran et al, 2018), CNN+PtrNet (Gong et al, 2016), LSTM+PtrNet (Gong et al, 2016), and ATTOrderNet (Cui et al, 2018). Here, except the random model, all of the baselines are based on neural networks, which are typically more competitive than traditional approaches (e.g., utilizing handcraft features).…”

“…To address the issue, ATTOrderNet (Cui et al, 2018) employs self-attention at the encoder to capture global dependencies regardless of an input sentence order. However, similar to traditional models, they also compress sentence vectors into a single fixed-length vector via average pooling.…”

“…More recently, state-of-the-art models aim to put randomly sorted sentences into a coherent paragraph with the correct order so that the whole sentences have the highest coherence probability (Vinyals et al, 2015a;Gong et al, 2016;Logeswaran et al, 2018;Cui et al, 2018). Unlike the SPO models, these models can perform the sentence set ordering (SSO) task for an entire paragraph based on the pointer network (Vinyals et al, 2015b).…”

Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications

“…propose training sentence ordering models to differentiate between the original order of a well-written text and a permuted sentence order. Cui et al (2018) continue in this paradigm, training an encoderdecoder network to read a series of sentences and reorder them for better coherence. Our goal is not to reorder a student's sentences, but to provide more detailed feedback on whether the right structures (e.g., steps) are present in the methodology.…”