T-BERTSum: Topic-Aware Text Summarization Based on BERT

Ma, Tinghuai; Pan, Qian; Rong, Huan; Qian, Yurong; Tian, Yuan; Al-Nabhan, Najla

doi:10.1109/tcss.2021.3088506

Cited by 68 publications

(17 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…First Story Detection UMASS [10], LSH-UMASS [11], PAR-UMASS [12], K-Term [13], Rel-EFSD [1] Rel-EFSD [1] Question Answering PARALEX [14], SEMPRE [15], ParSEM-PRE MemNets [14], BERT [16], T5 [8], AL-BERT [17], DistiBERT [18], GPT-3 [19] BERT [16] Information Extraction Ollie [20], ReVerb [21], Open-IE [22], Tex-tRunner [23], RelNoun [24], CALMIE [25] ReVerb [21] Summarization T-BERTSum [26], PEGASUS [27], T5 [8] T5 [8] Fake News Detection TriFN [28], FakeNewsTracker [29], Tree CRF [30] BERT [16]…”

Section: Mining Task Relevant Methods News Monitormentioning

confidence: 99%

News Monitor: A Framework for Exploring News in Real-Time

Panagiotou

Saravanou

Gunopulos

2021

Data

View full text Add to dashboard Cite

News articles generated by online media are a major source of information. In this work, we present News Monitor, a framework that automatically collects news articles from a wide variety of online news portals and performs various analysis tasks. The framework initially identifies fresh news (first stories) and clusters articles about the same incidents. For every story, at first, it extracts all of the corresponding triples and, then, it creates a knowledge base (KB) using open information extraction techniques. This knowledge base is then used to create a summary for the user. News Monitor allows for the users to use it as a search engine, ask their questions in their natural language and receive answers that have been created by the state-of-the-art framework BERT. In addition, News Monitor crawls the Twitter stream using a dynamic set of “trending” keywords in order to retrieve all messages relevant to the news. The framework is distributed, online and performs analysis in real-time. According to the evaluation results, the fake news detection techniques utilized by News Monitor allow for a F-measure of 82% in the rumor identification task and an accuracy of 92% in the stance detection tasks. The major contribution of this work can be summarized as a novel real-time and scalable architecture that combines various effective techniques under a news analysis framework.

show abstract

Section: Mining Task Relevant Methods News Monitormentioning

confidence: 99%

News Monitor: A Framework for Exploring News in Real-Time

Panagiotou

Saravanou

Gunopulos

2021

Data

View full text Add to dashboard Cite

show abstract

“…Farahani [40] mentioned two approaches to deal with the task of text summarization using mT5 model and ParsBERT model and got good results on a data set named pn-summary for Persian abstractive text summarization. Ma [41] proposed a pre-trained model T-BERTSum for text summarization, which captures the key words of the topic information of social media, understands the meaning of the sentence and judges the topic of the message discussion, and then generates a high-quality section Summary. Kerui [42] uses BERT, Seq2seq and reinforcement learning to form a text summary model.…”

Section: Related Workmentioning

confidence: 99%

“…In summary, in our work, social media (such as Twitter), pre-trained models, and text summaries (including extractive and abstractive methods) are the three key elements of event summarization. From the above literature review, the text summaries of specific event tweets on social media have gradually become one of the most popular summarization research topics in recent years [1][2][3]41,42]. However, extractive summarization cannot obtain many key sentences in the tweet-form data, resulting in unsatisfactory quality of the generated summaries.…”

Section: Related Workmentioning

confidence: 99%

“…Furthermore, due to our task requires extensive domain knowledge beyond the meaning of individual words and sentences, in this work we limit our focus to abstractive summarization algorithms. On the other hand, in the recent references of text summarization, there are more and more work using pre-training language models [35][36][37][38][39][40][41][42][43][44]. Among them, BERT is the pre-trained model mainly used by most research teams.…”

Section: Related Workmentioning

confidence: 99%

“…Among them, BERT is the pre-trained model mainly used by most research teams. Most researchers use the improved BERT model architecture in their work [36,37,[39][40][41]. However, because the sentences in the pre-training data set of BERT have correct grammar, BERT does not perform well for tweets with less grammar.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Event Monitoring and Intelligence Gathering Using Twitter Based Real-Time Event Summarization and Pre-Trained Model Techniques

Lee

Yang

Chen³

et al. 2021

Applied Sciences

View full text Add to dashboard Cite

Recently, an emerging application field through Twitter messages and algorithmic computation to detect real-time world events has become a new paradigm in the field of data science applications. During a high-impact event, people may want to know the latest information about the development of the event because they want to better understand the situation and possible trends of the event for making decisions. However, often in emergencies, the government or enterprises are usually unable to notify people in time for early warning and avoiding risks. A sensible solution is to integrate real-time event monitoring and intelligence gathering functions into their decision support system. Such a system can provide real-time event summaries, which are updated whenever important new events are detected. Therefore, in this work, we combine a developed Twitter-based real-time event detection algorithm with pre-trained language models for summarizing emergent events. We used an online text-stream clustering algorithm and self-adaptive method developed to gather the Twitter data for detection of emerging events. Subsequently we used the Xsum data set with a pre-trained language model, namely T5 model, to train the summarization model. The Rouge metrics were used to compare the summary performance of various models. Subsequently, we started to use the trained model to summarize the incoming Twitter data set for experimentation. In particular, in this work, we provide a real-world case study, namely the COVID-19 pandemic event, to verify the applicability of the proposed method. Finally, we conducted a survey on the example resulting summaries with human judges for quality assessment of generated summaries. From the case study and experimental results, we have demonstrated that our summarization method provides users with a feasible method to quickly understand the updates in the specific event intelligence based on the real-time summary of the event story.

show abstract

A Lightweight Global Taxonomy Induction System for E-Commerce Concept Labels

Kejriwal

Shen

2023

Lecture Notes in Social Networks

View full text Add to dashboard Cite

T-BERTSum: Topic-Aware Text Summarization Based on BERT

Cited by 68 publications

References 29 publications

News Monitor: A Framework for Exploring News in Real-Time

News Monitor: A Framework for Exploring News in Real-Time

Event Monitoring and Intelligence Gathering Using Twitter Based Real-Time Event Summarization and Pre-Trained Model Techniques

A Lightweight Global Taxonomy Induction System for E-Commerce Concept Labels

Contact Info

Product

Resources

About