Extractive Summarization of Long Documents by Combining Global and Local Context

Xiao, Wen; Carenini, Giuseppe

doi:10.18653/v1/d19-1298

Cited by 132 publications

(106 citation statements)

References 28 publications

Supporting

Mentioning

105

Contrasting

Unclassified

Order By: Relevance

“…3. Multiple turns Similar to long document summarization (Xiao and Carenini, 2019), conversations with many utterances contain more information to be processed, thus harder to be summarized.…”

Section: Challenges In Dialog Summarizationmentioning

confidence: 99%

Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization

Chen¹,

Yang²

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

101

View full text Add to dashboard Cite

Text summarization is one of the most challenging and interesting problems in NLP. Although much attention has been paid to summarizing structured text like news reports or encyclopedia articles, summarizing conversations-an essential part of humanhuman/machine interaction where most important pieces of information are scattered across various utterances of different speakersremains relatively under-investigated. This work proposes a multi-view sequence-tosequence model by first extracting conversational structures of unstructured daily chats from different views to represent conversations and then utilizing a multi-view decoder to incorporate different views to generate dialogue summaries. Experiments on a large-scale dialogue summarization corpus demonstrated that our methods significantly outperformed previous state-of-the-art models via both automatic evaluations and human judgment. We also discussed specific challenges that current approaches faced with this task. We have publicly released our code at https://github.com/GT-SALT/ Multi-View-Seq2Seq.

show abstract

Section: Challenges In Dialog Summarizationmentioning

confidence: 99%

Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization

Chen¹,

Yang²

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

101

View full text Add to dashboard Cite

show abstract

“…Timeline summarization task is firstly proposed by Allan and et al [2001], in this paper, they propose a method that extracts a single sentence from each event within a news topic. Later, a series of works [Yan et al, 2011b;Yan et al, 2011a;Yan et al, 2012] further investigate the timeline summarization task, and all of them are based on conventional learning method to extract sentences from the timeline data. For instance, [Yan et al, 2011b] formulate the timeline summarization task as a balanced optimization problem via iterative substitution.…”

Section: Timeline Summarizationmentioning

confidence: 99%

“…They also conduct experiments on arXiv and PubMed datasets, and their model outperforms the baseline methods on these datasets. [Xiao and Carenini, 2019] propose an extractive method for this task using both the global context of the whole document and the local context within the current topic, and this method achieves state-of-the-art performance on the previous two datasets.…”

Section: Extreme Long Document Summarizationmentioning

confidence: 99%

From Standard Summarization to New Tasks and Beyond: Summarization with Manifold Information

Gao

Chen

Zhang

et al. 2020

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Text summarization is the research area aiming at creating a short and condensed version of the original document, which conveys the main idea of the document in a few words. This research topic has started to attract the attention of a large community of researchers, and it is nowadays counted as one of the most promising research areas. In general, text summarization algorithms aim at using a plain text document as input and then output a summary. However, in real-world applications, most of the data is not in a plain text format. Instead, there is much manifold information to be summarized, such as the summary for a web page based on a query in the search engine, extreme long document (e.g. academic paper), dialog history and so on. In this paper, we focus on the survey of these new summarization tasks and approaches in the real-world application.

show abstract

“…More recently, further work along this line started to incorporate discourse structures into supervised summarization with the goal to better leverage the (linguistic) structure of a document. Xiao and Carenini (2019) and Cohan et al (2018) thereby use the natural structure of scientific papers (i.e. sections) to improve the inputs of the sequence models, better encoding long documents using a structural prior.…”

Section: Discourse and Summarizationmentioning

confidence: 99%

Do We Really Need That Many Parameters In Transformer For Extractive Summarization? Discourse Can Help !

Xiao¹,

Huber²,

Carenini³

2020

Proceedings of the First Workshop on Computational Approaches to Discourse

Self Cite

View full text Add to dashboard Cite

The multi-head self-attention of popular transformer models is widely used within Natural Language Processing (NLP), including for the task of extractive summarization. With the goal of analyzing and pruning the parameterheavy self-attention mechanism, there are multiple approaches proposing more parameterlight self-attention alternatives. In this paper, we present a novel parameter-lean self-attention mechanism using discourse priors. Our new tree self-attention is based on document-level discourse information, extending the recently proposed "Synthesizer" framework with another lightweight alternative. We show empirical results that our tree self-attention approach achieves competitive ROUGE-scores on the task of extractive summarization. When compared to the original single-head transformer model, the tree attention approach reaches similar performance on both, EDU and sentence level, despite the significant reduction of parameters in the attention component. We further significantly outperform the 8-head transformer model on sentence level when applying a more balanced hyper-parameter setting, requiring an order of magnitude less parameters 1 .

show abstract

Extractive Summarization of Long Documents by Combining Global and Local Context

Cited by 132 publications

References 28 publications

Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization

Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization

From Standard Summarization to New Tasks and Beyond: Summarization with Manifold Information

Do We Really Need That Many Parameters In Transformer For Extractive Summarization? Discourse Can Help !

Contact Info

Product

Resources

About