Controlling Length in Abstractive Summarization Using a Convolutional Neural Network

Liu, Yizhu; Luo, Zhiyi; Zhu, Kenny Q.

doi:10.18653/v1/d18-1444

Cited by 50 publications

(59 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Other hyperparameters of models and optimization methods used in our experiments are summarized in Table 1. We halve the word embedding size, hidden state size, and the number of layers of LC from the original setting of Liu et al (2018). This is because avoiding out-of-memory error on our GPU when applying MRT, and GOLC, and our objective of the experiments with LC is the evaluation of length control ability of each optimization method.…”

Section: Optimization Methods To Be Comparedmentioning

confidence: 99%

“…Other methods for the improvement of abstractive summarization models include use of existing summaries as soft templates with a source text and extraction of actual fact descriptions from a source text . Although summary length control of abstractive summarization has been studied, previous studies focus on incorporation of a length controlling method to neural abstractive summarization models (Kikuchi et al, 2016;Fan et al, 2018;Liu et al, 2018;Fevry and Phang, 2018;Schumann, 2018). In contrast, our research focuses on a global optimization method.…”

Section: Related Workmentioning

confidence: 99%

“…We note that the length of a summary is the number of words in the summary in Liu et al (2018), while the length of a summary is the number of characters in the summary in Kikuchi et al (2016),…”

Section: Cnn Based Model (Lc)mentioning

confidence: 99%

“…To this problem, Kikuchi et al (2016) and Liu et al (2018) proposed abstractive summarization models with a capability of summary length control. One is an LSTM based summarization model, and the other is a CNN based one.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Global Optimization under Length Constraint for Neural Text Summarization

Makino¹,

Iwakura²,

Takamura³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

We propose a global optimization method under length constraint (GOLC) for neural text summarization models. GOLC increases the probabilities of generating summaries that have high evaluation scores, ROUGE in this paper, within a desired length. We compared GOLC with two optimization methods, a maximum log-likelihood and a minimum risk training, on CNN/Daily Mail and a Japanese single document summarization data set of The Mainichi Shimbun Newspapers. The experimental results show that a state-ofthe-art neural summarization model optimized with GOLC generates fewer overlength summaries while maintaining the fastest processing speed; only 6.70% overlength summaries on CNN/Daily and 7.8% on long summary of Mainichi, compared to the approximately 20% to 50% on CNN/Daily Mail and 10% to 30% on Mainichi with the other optimization methods. We also demonstrate the importance of the generation of in-length summaries for post-editing with the dataset Mainich that is created with strict length constraints. The experimental results show approximately 30% to 40% improved post-editing time by use of inlength summaries.

show abstract

Section: Optimization Methods To Be Comparedmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Cnn Based Model (Lc)mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Global Optimization under Length Constraint for Neural Text Summarization

Makino¹,

Iwakura²,

Takamura³

et al. 2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…LenInit initializes the LSTM cell of the decoder with the embedding depending on the scalar value of the desired length. Liu et al (2018) incorporated such scalar values into the initial state of the decoder in a CNN encoder-decoder. These approaches deal with any length but it is reasonable to incorporate the distance to the desired terminal position into each decoding step such as in LenEmb.…”

Section: Introductionmentioning

confidence: 99%

Positional Encoding to Control Output Sequence Length

Takase¹,

Okazaki²

2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

Neural encoder-decoder models have been successful in natural language generation tasks. However, real applications of abstractive summarization must consider additional constraint that a generated summary should not exceed a desired length. In this paper, we propose a simple but effective extension of a sinusoidal positional encoding (Vaswani et al., 2017) to enable neural encoder-decoder model to preserves the length constraint. Unlike in previous studies where that learn embeddings representing each length, the proposed method can generate a text of any length even if the target length is not present in training data. The experimental results show that the proposed method can not only control the generation length but also improve the ROUGE scores.

show abstract