Concept-Based Label Distribution Learning for Text Classification

Software Defect Detection (SDD) has always been critical to the development life cycle. A stable defect detection system can not only alleviate the workload of software testers but also enhance the overall efficiency of software development. Researchers have recently proposed various artificial intelligence-based SDD methods and achieved significant advancements. However, these methods still exhibit limitations in terms of reliability and usability. Therefore, we introduce MSDD-(IA)3, a novel framework leveraging the pre-trained CodeT5+ and (IA)3 for parameter-efficient multi-classification SDD. This framework constructs a detection model based on pre-trained CodeT5+ to generate code representations while capturing defect-prone features. Considering the high overhead of pre-trained LLMs, we injects (IA)3 vectors into specific layers, where only these injected parameters are updated to reduce the training cost. Furthermore, leveraging the properties of the pre-trained CodeT5+, we design a novel feature sequence that enriches the input data through the combination of source code with Natural Language (NL)-based expert metrics. Our experimental results on 64K real-world Python snippets show that MSDD-(IA)3 demonstrates superior performance compared to state-of-the-art SDD methods, including PM2-CNN, in terms of F1-weighted, Recall-weighted, Precision-weighted, and Matthews Correlation Coefficient. Notably, the training parameters of MSDD-(IA)3 are only 0.04% of those of the original CodeT5+. Our experimental data and code can be available at (https://gitee.com/wxyzjp123/msdd-ia3/).

Section: Deep Learning-based Software Defect Detectionmentioning

confidence: 99%

Parameter-Efficient Multi-classification Software Defect Detection Method Based on Pre-trained LLMs

Wang,

Lu,

Yang

et al. 2024

“…According to the adaptive neuro-fuzzy inference system ( ANFIS ) successful application reports and the aim of this study, it is worth scrutinizing the potential and applicability of this model in various fields [23][24][25][26][27]. Several studies have recently been developed employing different types of machine learning-based algorithms [28][29][30][31][32][33][34][35][36][37][38][39][40][41]. The application of theANFISwas reported successfully in several publications in single or hybrid forms [42].…”

Section: Literature Reviewmentioning

confidence: 99%

Optimization-Based Fuzzy System Application on Deformation of Geogrid-Reinforced Soil Structures

Dou

2024

When it comes to the geosynthetic-reinforced soil structures’ structural design, the analysis of deformation is of the highest relevance. In spite of this, the academic literature lays a substantial amount of attention on the potential of artificial intelligence approaches in efficiently addressing the many challenges that are faced in geotechnical engineering. The investigation of the possible use of approaches based on machine learning for the purpose of forecasting the geogrid-reinforced soil structures’ deformation ($${\text{Dis}}$$ Dis ) was the major focus of this study. These efforts are made to reduce the time and cost of numerical modeling. This study aimed to enable computers to learn patterns and insights from data to make accurate predictions or decisions about unseen data. This paper introduces novel systems that coupled the Beluga whale optimizer ($${\text{BWH}}$$ BWH ), Henry gas solubility optimization ($${\text{HGSO}}$$ HGSO ), gannet optimization algorithm ($${\text{GOA}}$$ GOA ), and Harris hawks optimizer ($${\text{HHO}}$$ HHO ) with adaptive neuro-fuzzy inference system ($${\text{ANFIS}}$$ ANFIS ). A dataset was created by gathering 166 finite element analyses accomplished in the literature. Between four $${\text{ANFIS}}$$ ANFIS systems, the integrated one with $${\text{GOA}}$$ GOA got the largest accuracy value, accounting for 0.9841 and 0.9895 in the train and test stages, better than $${\text{ANF}}_{{{\text{BWH}}}}$$ ANF BWH , followed by $${\text{ANF}}_{{{\text{HH}}}}$$ ANF HH . It was seen from $${\text{U}}_{{{95}}} { }$$ U 95 that the $${\text{ANF}}_{{{\text{GOA}}}} { }$$ ANF GOA scenario exhibits the least level of uncertainty in comparison to other models, hence demonstrating its greater capacity for generalization. Between four $${\text{ANFIS}}$$ ANFIS systems, the most accurate system with the lowest $${\text{OBJ}}$$ OBJ value is $${\text{ANF}}_{{{\text{GOA}}}}$$ ANF GOA at 2.6098, followed by $${\text{ANF}}_{{{\text{BWH}}}}$$ ANF BWH at 2.8002. Sensitivity analysis depicts that removing the surcharge $$\left( q \right)$$ q parameter from the input group has a considerable negative impact on output accuracy.

“…With the development of deep learning and pre-trained language models, text classification has gained great success [1][2][3][4][5]. Generally, training deep-learning text classification models require a large amount of labeled data to achieve competitive performance.…”

Section: Introductionmentioning

confidence: 99%

SEML: Self-Supervised Information-Enhanced Meta-learning for Few-Shot Text Classification

Huang

et al. 2023

Self Cite

Training a deep-learning text classification model usually requires a large amount of labeled data, yet labeling data are usually labor-intensive and time-consuming. Few-shot text classification focuses on predicting unknown samples using only a few labeled samples. Recently, metric-based meta-learning methods have achieved promising results in few-shot text classification. They use episodic training in labeled samples to enhance the model’s generalization ability. However, existing models only focus on learning from a few labeled samples but neglect to learn from a large number of unlabeled samples. In this paper, we exploit the knowledge learned by the model in unlabeled samples to improve the generalization performance of the meta-network. Specifically, we introduce a novel knowledge distillation method that expands and enriches the meta-learning representation with self-supervised information. Meanwhile, we design a graph aggregation method that efficiently interacts the query set information with the support set information in each task and outputs a more discriminative representation. We conducted experiments on three public few-shot text classification datasets. The experimental results show that our model performs better than the state-of-the-art models in 5-way 1-shot and 5-way 5-shot cases.