Abstractive text summarization model based on BERT vectorization and bidirectional decoding

Abstract

Facing the pressure of the extremely inflated network information and data overload surplus, it is extremely important to locate “valuable information” efficiently and accurately. Text summarization technology in the field of Natural Language Processing (NLP) is an effective means to analyze and process network information. In this paper, we propose an abstractive text summarization model based on bidirectional encoder representations from transformers (BERT) vectorization and bidirectional decoding. The BERT is adopted to obtain a more global vector representation, which helps the subsequent encoder and decoder to fuse the full-text information to generate a summary with high generality. The decoding phase adopts a bidirectional decoding structure and combines the attention mechanism to maintain the bilateral decoding result to generate summaries. The bidirectional decoding structure can be fine-tuned according to the bidirectional results, which can overcome the tilt problem of the unidirectional structure, and the generated summaries are more consistent. The experimental results on the NLPCC2017 text summarization dataset show that the summaries generated by our model have the higher coherence at the word and sentence level, and the stronger generalization of the full text.