Bert-Based Feature Extraction for Long-Lived Bug Prediction in Floss: A Comparative Study

Gomes, Luiz Alberto Ferreira; Côrtes, Mário Lúcio; Torres, Ricardo da Silva

doi:10.2139/ssrn.4166555

Cited by 5 publications

(5 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Relation extraction and classification are conducted in the first and second phases respectively. The second phase involves fine-tuning the BERT model to perform tasks such as token/ sequence classification or question answering [139]. In ontology construction, the attention masks are added to the token IDs to fine-tune the model for text classification.…”

Section: Figure 21 Blstm -Cnn [132] [130]mentioning

confidence: 99%

“…In ontology construction, the attention masks are added to the token IDs to fine-tune the model for text classification. Gomes et al [139] explained that BERT replaces 15% of word sequences with a masked token, then leverages contextual information from the surrounding, unmasked words within the sequence to predict the original values of the masked words. The technical process involves introducing a classification layer over the output generated by the encoder.…”

Section: Figure 21 Blstm -Cnn [132] [130]mentioning

confidence: 99%

“…The technical process involves introducing a classification layer over the output generated by the encoder. The vector outputs are transformed into vocabulary dimensions by multiplying them with the embedding matrix and utilizing a softmax to compute word probabilities [139]. The BERT model demonstrated exceptional performance in word representation and comprehension of context-heavy texts.…”

Section: Figure 21 Blstm -Cnn [132] [130]mentioning

confidence: 99%

See 2 more Smart Citations

A Review of State of the Art Deep Learning Models for Ontology Construction

Zengeya,

Vincent Fonou-Dombeu

2024

IEEE Access

View full text Add to dashboard Cite

ABSRTACT Researchers are working towards automation of ontology construction to manage the evergrowing data on the web. Currently, there is a shift from the use of machine learning techniques towards exploration of deep learning models for ontology construction. Deep learning model are capable of extracting terms, entities, relations, and classifications, and perform axiom learning from the underutilized richness of web-based knowledge. There has been remarkable progress in automatic ontology creation using deep learning models since they can perform word embedding, long-term dependency acquisition, concept extraction from large corpora, and inference of abstracted relationships based on broad corpora. Despite their emerging importance, deep learning models remain underutilized in ontology construction, and there is no comprehensive review of their application in ontology learning. This paper presents a comprehensive review of existing deep-learning models for the construction of ontologies, the strength and the weaknesses presented by the deep learning models for ontology learning as well as promising directions to achieve a more robust deep learning models. The Deep Learning models reviewed include Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), Long-Short Term Memory (LSTMs), and Gated Recurrent Unit (GRU) as well as their ensembles. While these traditional deep learning models have achieved great success, one of their limitations is that they struggle to understand the meaning and order of data in sequences. CNNs and RNN-based models such as LSTMs and GRUs can be computationally expensive due to their large number of parameters or complex gating mechanisms. Furthermore, RNN models suffer from vanishing gradients, making it difficult to learn long-term relationships in sequences. Additionally, RNN-based models process information sequentially, limiting their ability to take advantage of powerful parallel computing hardware, slowing down training and inference, especially for long sequences. Consequently, there has been a shift towards Generative Pre-Trained (GPT) models and Bidirectional Encoder Representations from Transformers (BERT) models. This paper also reviewed the GPT-3, GPT-4, and the BERT models for extracting terms, entities, relations, and classifications. While GPT models excel in contextual understanding and flexibility, they fall short when handling domain-specific terminology and disambiguating complex relationships. Fine-tuning and domain-specific training data could minimize these shortcomings, and further enhance the performance of GPT in term and relation extraction tasks. On the other hand, the BERT models excel in comprehending context-heavy texts, but struggles with higher-level abstraction and inference tasks due to a lack of explicit semantic knowledge, thus necessitating inference for unspecified relationships. The paper recommends further research on deep learning models for ontology alignment and merging. Also, the ensembling of deep learning models and the u...

show abstract

Section: Figure 21 Blstm -Cnn [132] [130]mentioning

confidence: 99%

Section: Figure 21 Blstm -Cnn [132] [130]mentioning

confidence: 99%

Section: Figure 21 Blstm -Cnn [132] [130]mentioning

confidence: 99%

See 1 more Smart Citation

A Review of State of the Art Deep Learning Models for Ontology Construction

Zengeya,

Vincent Fonou-Dombeu

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Both machine learning and deep learning have been employed to predict bug fixing times (Giger et al, 2010;Marks et al, 2011;Zhang et al, 2013;Habayeb et al, 2018;Lee et al, 2020;Gomes et al, 2022Gomes et al, , 2023. A summary of studies that conducted mining on bug management databases to achieve learning models related to bug fixing times is presented in Table 1, which is an extension of the research summarization by (Lee et al, 2020), with additions of the latest studies and our method.…”

Section: Predicting Bug Fixing Times and Severitymentioning

confidence: 99%

“…Furthermore, to the best of our knowledge, important words and characteristics to predict bug fixing time have yet to be determined. Although there has been a comparison of different text feature extraction approaches in terms of the accuracy of machine learning classifiers (Gomes et al, 2022(Gomes et al, , 2023, important concrete features have yet to be examined well. The influence of comments on bug fixing time is not well understood.…”

Section: Predicting Bug Fixing Times and Severitymentioning

confidence: 99%

Deep learning and gradient-based extraction of bug report features related to bug fixing time

Noyori

Washizaki

Fukazawa

et al. 2023

Front. Comput. Sci.

View full text Add to dashboard Cite

Bug reports typically contain detailed descriptions of failures, hints at the location of the corresponding defects, and discussions. Developers usually resolve bugs using comments in descriptions and discussions. The time to fix a bug varies greatly. Previous studies have investigated bug reports, but the influence of comments on bug fixing time is not well understood. This study adopts a convolutional neural network (CNN) and gradient-based visualization approach called Grad-cam to elucidate the impact of comments on bug fixing time and extract features. A feature represents an observed characteristic in a bug report when processing via deep learning. Specifically, CNN classifies bug reports, and then Grad-cam visualizes the decision basis of CNN by identifying the top 10 word sequences used in the prediction. Here, the features are major word sequences extracted by Grad-cam. In an experiment, the proposed method classified more than 36,000 actual bug reports from Bugzilla with an accuracy of 75%–80%. Additionally, the visualization highlighted differences in the stack trace and word abstraction by bug fixing time. Bug reports with short bug fixing times are concrete, whereas those with a long bug fixing time are abstract.

show abstract