Untitled

Baziotis, Christos; Androutsopoulos, Ion; Konstas, Ioannis; Potamianos, Alexandros

doi:10.18653/v1/n19-1071

Cited by 41 publications

(38 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dataset. The same as Baziotis et al (2019), we train our model on the Gigaword corpus (GIGA, Rush et al, 2015). However, we used only 30K sentences randomly picked from sentences with less than 50 words for the training of EALM.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Q-learning with Language Model for Edit-based Unsupervised Summarization

Kohita

Wachi

Zhao

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Unsupervised methods are promising for abstractive textsummarization in that the parallel corpora is not required. However, their performance is still far from being satisfied, therefore research on promising solutions is on-going. In this paper, we propose a new approach based on Q-learning with an edit-based summarization. The method combines two key modules to form an Editorial Agent and Language Model converter (EALM). The agent predicts edit actions (e.t., delete, keep, and replace), and then the LM converter deterministically generates a summary on the basis of the action signals. Qlearning is leveraged to train the agent to produce proper edit actions. Experimental results show that EALM delivered competitive performance compared with the previous encoderdecoder-based methods, even with truly zero paired data (i.e., no validation set). Defining the task as Q-learning enables us not only to develop a competitive method but also to make the latest techniques in reinforcement learning available for unsupervised summarization. We also conduct qualitative analysis, providing insights into future study on unsupervised summarizers. 1

show abstract

Section: Methodsmentioning

confidence: 99%

“…We followed Baziotis et al (2019) in the evaluation as well, using the test set consisting of the GIGA (1897 sentences) and DUC datasets (DUC3 with 624 sentences, DUC4 with 500 sentences; Over et al 2007).…”

Section: Methodsmentioning

confidence: 99%

Q-learning with Language Model for Edit-based Unsupervised Summarization

Kohita

Wachi

Zhao

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

show abstract

“…Recently, Ren et al (2019) used posterior regularization for unsupervised NMT, by employing an SMT model, which is robust to noisy data, as a prior over a neural TM to guide it in the iterative back-translation process. Finally, LMs have been used in a similar fashion as priors over latent text sequences in discrete latent variable models (Miao and Blunsom, 2016;Havrylov and Titov, 2017;Baziotis et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

Language Model Prior for Low-Resource Neural Machine Translation

Baziotis¹,

Haddow²,

Birch³

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Self Cite

View full text Add to dashboard Cite

The scarcity of large parallel corpora is an important obstacle for neural machine translation. A common solution is to exploit the knowledge of language models (LM) trained on abundant monolingual data. In this work, we propose a novel approach to incorporate a LM as prior in a neural translation model (TM). Specifically, we add a regularization term, which pushes the output distributions of the TM to be probable under the LM prior, while avoiding wrong predictions when the TM "disagrees" with the LM. This objective relates to knowledge distillation, where the LM can be viewed as teaching the TM about the target language. The proposed approach does not compromise decoding speed, because the LM is used only at training time, unlike previous work that requires it during inference. We present an analysis on the effects that different methods have on the distributions of the TM. Results on two low-resource machine translation datasets show clear improvements even with limited monolingual data.

show abstract

“…The model described in Wang and Lee (2018) utilizes seq2seq auto-encoding coupled with adversarial training and reinforcement learning. Fevry and Phang (2018) and Baziotis et al (2019) focus on sentence summarization (i.e., compression). Chu and Liu (2019) proposes yet another denoising auto-encoding based model in multi-document summarization domain.…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers

Xu¹,

Zhang²,

Wu³

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Unsupervised extractive document summarization aims to select important sentences from a document without using labeled summaries during training. Existing methods are mostly graph-based with sentences as nodes and edge weights measured by sentence similarities. In this work, we find that transformer attentions can be used to rank sentences for unsupervised extractive summarization. Specifically, we first pre-train a hierarchical transformer model using unlabeled documents only. Then we propose a method to rank sentences using sentence-level self-attentions and pre-training objectives. Experiments on CNN/DailyMail and New York Times datasets show our model achieves state-of-the-art performance on unsupervised summarization. We also find in experiments that our model is less dependent on sentence positions. When using a linear combination of our model and a recent unsupervised model explicitly modeling sentence positions, we obtain even better results.

show abstract

Untitled

Cited by 41 publications

References 24 publications

Q-learning with Language Model for Edit-based Unsupervised Summarization

Q-learning with Language Model for Edit-based Unsupervised Summarization

Language Model Prior for Low-Resource Neural Machine Translation

Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers

Contact Info

Product

Resources

About