What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment

Abstract

We propose an end-to-end, domainindependent neural encoder-aligner-decoder model for selective generation, i.e., the joint task of content selection and surface realization. Our model first encodes a full set of over-determined database event records via an LSTM-based recurrent neural network, then utilizes a novel coarse-to-fine aligner to identify the small subset of salient records to talk about, and finally employs a decoder to generate free-form descriptions of the aligned, selected records. Our model achieves the best selection and generation results reported to-date (with 59% relative improvement in generation) on the benchmark WEATHER-GOV dataset, despite using no specialized features or linguistic resources. Using an improved k-nearest neighbor beam filter helps further. We also perform a series of ablations and visualizations to elucidate the contributions of our key model components. Lastly, we evaluate the generalizability of our model on the ROBOCUP dataset, and get results that are competitive with or better than the state-of-the-art, despite being severely data-starved.

Coarse-to-Fine Attention Models for Document Summarization

“…Inspired by the discount factor from RL, we slightly modify the total reward: instead of simply taking the sum, we can scale later rewards with a discount factor γ, giving total reward T s=t γ s−t r s for the stochastic hard attention node a t . We found 1 The term coarse-to-fine attention has previously been introduced in the literature (Mei et al, 2016). However, their idea is different: they use coarse attention to reweight the fine attention computed over the entire input.…”

Neural Networks for Information Retrieval

“…Targeting this newly emerging demand, some models have been proposed to respond by generating natural language replies on the y, rather than by (re)ranking a xed set of items or extracting passages from existing pages. Examples are conversational and dialog systems [7,34,54] or machine reading and question answering tasks where the model either infers the answer from unstructured data, like textual documents that do not necessarily feature the answer literally [21,22,46,56], or generates natural language given structured data, like data from knowledge graphs or from external memories [1,18,33,37,40].…”

Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)

“…proposed a review network to the image captioning, which reviews all the information encoded by the encoder and produces a compact thought vector. Mei et al (2015) proposed RNN encoder-decoderbased model by using two attention layers to jointly train content selection and surface realization. More close to our work, Wen et al (2016b) proposed an attentive encoder-decoder based generator which computed the attention mechanism over the slot-value pairs.…”

“…To ensure that the generated utterance representing the intended meaning of the given DA, the previous RNN-based models were further conditioned on a 1-hot vector representation of the DA. More recently, Encoder-Decoder networks , especially the attentional based models (Wen et al, 2016b;Mei et al, 2015) have been explored to solve the NLG tasks. The Attentional RNN EncoderDecoder (ARED) based approaches have also shown improved performance on a variety of tasks, e.g., image captioning , text summarization (Rush et al, 2015;.…”