Towards State-of-the-art Baselines for Vietnamese Multi-document Summarization

Nguyen, Minh-Tien; Nguyen, Hoang-Diep; Nguyen, Thi-Hai-Nang; Nguyen, Van-Hau

doi:10.1109/kse.2018.8573420

Cited by 5 publications

(1 citation statement)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of these focus on inspecting extractive summarization. The researchers (Nguyen et al, 2018) com-pared a wide range of extractive methods, including unsupervised ranking methods (e.g., LexRank, LSA, KL-divergence), supervised learning methods using TF-IDF and classifiers (e.g., Support Vector Machine, AdaBoost, Learning-2-rank), and deep learning methods (e.g., Convolutional Neural Network, Long-Short Term Memory). Similarly, the authors (Nguyen et al, 2019) also evaluated the extractive methods on their own dataset, which was released publicly as a benchmark for future studies.…”

Section: Related Workmentioning

confidence: 99%

ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation

Phan¹,

Tran²,

Nguyen³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present ViT5, a pretrained Transformerbased encoder-decoder model for the Vietnamese language.With T5-style selfsupervised pretraining, ViT5 is trained on a large corpus of high-quality and diverse Vietnamese texts. We benchmark ViT5 on two downstream text generation tasks, Abstractive Text Summarization and Named Entity Recognition. Although Abstractive Text Summarization has been widely studied for the English language thanks to its rich and large source of data, there has been minimal research into the same task in Vietnamese, a much lower resource language. In this work, we perform exhaustive experiments on both Vietnamese Abstractive Summarization and Named Entity Recognition, validating the performance of ViT5 against many other pretrained Transformer-based encoderdecoder models. Our experiments show that ViT5 significantly outperforms existing models and achieves state-of-the-art results on Vietnamese Text Summarization. On the task of Named Entity Recognition, ViT5 is competitive against previous best results from pretrained encoder-based Transformer models. Further analysis shows the importance of context length during the self-supervised pretraining on downstream performance across different settings.

show abstract

Section: Related Workmentioning

confidence: 99%