Unsupervised extractive multi-document summarization method based on transfer learning from BERT multi-task fine-tuning

Lamsiyah, Salima; Mahdaouy, Abdelkader El; Ouatik, Saïd El Alaoui; Espinasse, Bernard

doi:10.1177/0165551521990616

Cited by 27 publications

(10 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the process of transfer learning, there are two models for fine-tuning. The first method is to freeze all convolutional feature extraction layers of the pre-trained model and only perform fine-tuning training operations on the classification layer; the second method is to fine tune all convolutional feature extraction and classification layers of the pre-trained model ( Lamsiyah et al, 2023 ).…”

Section: Methodsmentioning

confidence: 99%

Defect identification of bare printed circuit boards based on Bayesian fusion of multi-scale features

Han,

Li,

Wang

et al. 2024

PeerJ Computer Science

View full text Add to dashboard Cite

The aim of this article is to propose a defect identification method for bare printed circuit boards (PCB) based on multi-feature fusion. This article establishes a description method for various features of grayscale, texture, and deep semantics of bare PCB images. First, the multi-scale directional projection feature, the multi-scale grey scale co-occurrence matrix feature, and the multi-scale gradient directional information entropy feature of PCB were extracted to build the shallow features of defect images. Then, based on migration learning, the feature extraction network of the pre-trained Visual Geometry Group16 (VGG-16) convolutional neural network model was used to extract the deep semantic feature of the bare PCB images. A multi-feature fusion method based on principal component analysis and Bayesian theory was established. The shallow image feature was then fused with the deep semantic feature, which improved the ability of feature vectors to characterize defects. Finally, the feature vectors were input as feature sequences to support vector machines for training, which completed the classification and recognition of bare PCB defects. Experimental results show that the algorithm integrating deep features and multi-scale shallow features had a high recognition rate for bare PCB defects, with an accuracy rate of over 99%.

show abstract

Section: Methodsmentioning

confidence: 99%

Defect identification of bare printed circuit boards based on Bayesian fusion of multi-scale features

Han,

Li,

Wang

et al. 2024

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…Fine-tuning is a technique used to adjust the model to a new dataset. MBART50 is one of the pre-trained so that fine-tuning will be done [33], [34].…”

Section: Fine-tunning Modelmentioning

confidence: 99%

“…Unlike extractive summarization, which can produce poor sentences, abstractive summarization can produce grammatically correct sentences [6], [16], [17]. The abstractive method paraphrases and rearranges sentences into a summary [18], [19]. In this research, we will use the abstractive summarization method.…”

Section: Introductionmentioning

confidence: 99%

Indonesian News Text Summarization Using MBART Algorithm

Astuti,

Muljono,

Sutriawan

2024

SJI

View full text Add to dashboard Cite

Purpose: Technology advancements have led to the production of a large amount of textual data. There are numerous locations where one can find textual information sources, including blogs, news portals, and websites. Kompas, BBC, Liputan 6, CNN, and other news portals are a few websites that offer news in Indonesian. The purpose of this study was to explore the effectiveness of using mBART in text summarization for Bahasa Indonesia.Methods: This study uses mBART, a transformer architecture, to perform fine-tuning to generate news article summaries in Bahasa Indonesia. Evaluation was conducted using the ROUGE method to assess the quality of the summaries produced.Results: Evaluation using the ROUGE metric showed better results, with ROUGE-1 of 35.94, ROUGE-2 of 16.43, and ROUGE-L of 29.91. However, the performance of the model is still not optimal compared to existing models in text summarization for another language.Novelty: The novelty of this research lies in the use of mBART for text summarization, specifically adapted for Bahasa Indonesia. In addition, the findings also contribute to understanding the challenges and opportunities of improving text summarization techniques in the Indonesian context.

show abstract

“…Supervised methods need a significant volume of data, while unsupervised techniques require no educational data [2,5,13,14,18]. Promising results are obtained by transferring learning using pre-trained word embeddings within the scope of supervised learning [19].…”

Section: Introductionmentioning

confidence: 99%

A new multi-document summarisation approach using saplings growing-up optimisation algorithms: Simultaneously optimised coverage and diversity

Hark

Uçkan

Karcı

2022

Journal of Information Science

View full text Add to dashboard Cite

Automatic text summarisation is obtaining a subset that accurately represents the main text. A quality summary should contain the maximum amount of information while avoiding redundant information. Redundancy is a severe deficiency that causes unnecessary repetition of information within sentences and should not occur in summarisation studies. Although many optimisation-based text summarisation methods have been proposed in recent years, there exists a lack of research on the simultaneous optimisation of scope and redundancy. In this context, this study presents an approach in which maximum coverage and minimum redundancy, which form the two key features of a rich summary, are modelled as optimisation targets. In optimisation-based text summarisation studies, different conflicting objectives are generally weighted or formulated and transformed into single-objective problems. However, this transformation can directly affect the quality of the solution. In this study, the optimisation goals are met simultaneously without transformation or formulation. In addition, the multi-objective saplings growing-up algorithm (MO-SGuA) is implemented and modified for text summarisation. The presented approach, called Pareto optimal, achieves an optimal solution with simultaneous optimisation. Experimentation with the MO-SGuA method was tested using open-access (document understanding conference; DUC) data sets. Performance success of the MO-SGuA approach was calculated using the recall-oriented understudy for gisting evaluation (ROUGE) metrics and then compared with the competitive practices used in the literature. Testing achieved a 26.6% summarisation result for the ROUGE-2 metric and 65.96% for ROUGE-L, which represents an improvement of 11.17% and 20.54%, respectively. The experimental results showed that good-quality summaries were achieved using the proposed approach.

show abstract

Unsupervised extractive multi-document summarization method based on transfer learning from BERT multi-task fine-tuning

Cited by 27 publications

References 49 publications

Defect identification of bare printed circuit boards based on Bayesian fusion of multi-scale features

Defect identification of bare printed circuit boards based on Bayesian fusion of multi-scale features

Indonesian News Text Summarization Using MBART Algorithm

A new multi-document summarisation approach using saplings growing-up optimisation algorithms: Simultaneously optimised coverage and diversity

Contact Info

Product

Resources

About