Automatic detection of Long Method and God Class code smells through neural source code embeddings

Kovačević, Aleksandar; Slivka, Jelena; Vidaković, Dragan; Grujić, Katarina-Glorija; Luburić, Nikola; Prokić, Simona; Sladić, Goran

doi:10.1016/j.eswa.2022.117607

Cited by 37 publications

(56 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ML classifier trained using source code metrics as features outperformed the ML classifier trained using CodeT5 embeddings on both tasks and achieved the F-measure of 0.87 on the Long Method task and 0.91 on the Large Class task. This result differs from our findings in [10], where we found CuBERT [21] source code embeddings better capture the semantics needed to tackle the code smell detection problem. We may attribute this difference to the datasets used in the studies.…”

Section: Introductioncontrasting

confidence: 99%

“…In the DL approach, the relevant features are automatically inferred and hold the potential to capture semantics we are currently unable to express through existing code metrics. In contrast, as we showed in [10], the metric extraction process may be brittle, time-consuming, and not scalable. Secondly, the CodeT5 approach enables us to leverage the power of transfer learning.…”

Section: Introductionmentioning

confidence: 92%

“…In this paper, we use the dataset presented in [13] annotated for the Long Method and Large Class smells as these smells frequently occur in code and have a high negative impact on the software product and developer productivity, as we explained in more detail in [10], [13].…”

Section: Target Code Smellsmentioning

confidence: 99%

“…Literature reviews on code smell detection unanimously call for a standardized benchmark dataset on which researchers could fairly compare different code smell detection approaches [6], [7], [8], [9]. In our previous work [10], we tackled the problem of developing a machine learning (ML)-based code smell detector. Like other researchers, we, unfortunately, found that the publicly available code smell datasets may be hard to reproduce [11], [12] and contain noisy labels due to annotators' inconsistent understand-ing of the code smells [12].…”

Section: Introductionmentioning

confidence: 99%

“…The dataset used in this study was labeled by applying a carefully designed annotation procedure that ensured quality labels and high annotator agreement. In contrast, in [10], we used the MLCQ dataset [11] annotated to understand how the developers of different backgrounds perceive code smells, which resulted in inconsistent labels due to a diverse understanding of the code smells.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Automatic detection of code smells using metrics and CodeT5 embeddings: a case study in C#

Kovačević¹,

Luburić²,

Slivka³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Code smells are code structures that harm the software’s quality. An obstacle to developing automatic detectors is the available datasets' limitations. Furthermore, researchers developed many solutions for Java while neglecting other programming languages. Recently, we created the code smell dataset for C# by following an annotation procedure inspired by the established annotation practices in Natural Language Processing. This paper evaluates Machine Learning (ML) code smell detection approaches on our novel dataset. We consider two feature representations to train ML models: (1) code metrics and (2) CodeT5 embeddings. This study is the first to consider the CodeT5 state-of-the-art neural source code embedding for code smell detection in C#. To prove the effectiveness of ML, we consider multiple metrics-based heuristics as alternatives. In our experiments, the best-performing approach was the ML classifier trained on code metrics (F-measure of 0.87 for Long Method and 0.91 for Large Class detection). However, the performance improvement over CodeT5 features is negligible if we consider the advantages of automatically inferring features. We showed that our model exceeds human performance and could be helpful to developers. To the best of our knowledge, this is the first study to compare the performance of automatic smell detectors against human performance.

show abstract

Section: Introductioncontrasting

confidence: 99%

Section: Introductionmentioning

confidence: 92%

Section: Target Code Smellsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Automatic detection of code smells using metrics and CodeT5 embeddings: a case study in C#

Kovačević¹,

Luburić²,

Slivka³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Code smell prioritization in object‐oriented software systems: A systematic literature review

Verma

Kumar

Verma

2023

J Software Evolu Process

View full text Add to dashboard Cite

Code smells indicate poor design and imperfection in coding style. They increase maintenance costs and degrade software quality. Detecting and mitigating these code smells reduce the effects of flaws present in the software design. However, to save time and effort, it is a good practice to prioritize critical code smells and resolve them first. There have been various empirical studies on code smell prioritization in the last decades. Although the research community has been continuously working on the prioritization of code smells, yet a systematic mapping of all these studies would provide a panorama of research work. With this motivation, we selected 39 primary studies published between May 2011 and May 2022 and classified them based on seven factors: (i) code smells, (ii) factors, (iii) evaluation techniques, (iv) tools, (v) ranking formulas, (vi) data sets used, and (vii) measurements used for validating the research work. This paper aims to outline and discuss the prioritization procedure for code smells in object‐oriented software systems. The issues and recommendations for every question act as input for the researchers and industrialists for further research. Some issues for further investigation are less coverage of architecture smells, limited ranking formulas, lack of automated evaluation techniques, and so forth.

show abstract

On the relative value of imbalanced learning for code smell detection

Zou

Keung

et al. 2023

Softw Pract Exp

View full text Add to dashboard Cite

SummaryMachine learning‐based code smell detection (CSD) has been demonstrated to be a valuable approach for improving software quality and enabling developers to identify problematic patterns in code. However, previous researches have shown that the code smell datasets commonly used to train these models are heavily imbalanced. While some recent studies have explored the use of imbalanced learning techniques for CSD, they have only evaluated a limited number of techniques and thus their conclusions about the most effective methods may be biased and inconclusive. To thoroughly evaluate the effect of imbalanced learning techniques for machine learning‐based CSD, we examine 31 imbalanced learning techniques with seven classifiers to build CSD models on four code smell data sets. We employ four evaluation metrics to assess the detection performance with the Wilcoxon signed‐rank test and Cliff's . The results show that (1) Not all imbalanced learning techniques significantly improve detection performance, but deep forest significantly outperforms the other techniques on all code smell data sets. (2) SMOTE (Synthetic Minority Over‐sampling TEchnique) is not the most effective technique for resampling code smell data sets. (3) The best‐performing imbalanced learning techniques and the top‐3 data resampling techniques have little time cost for code smell detection. Therefore, we provide some practical guidelines. First, researchers and practitioners should select the appropriate imbalanced learning techniques (e.g., deep forest) to ameliorate the class imbalance problem. In contrast, the blind application of imbalanced learning techniques could be harmful. Then, better data resampling techniques than SMOTE should be selected to preprocess the code smell data sets.

show abstract

Automatic detection of Long Method and God Class code smells through neural source code embeddings

Cited by 37 publications

References 36 publications

Automatic detection of code smells using metrics and CodeT5 embeddings: a case study in C#

Automatic detection of code smells using metrics and CodeT5 embeddings: a case study in C#

Code smell prioritization in object‐oriented software systems: A systematic literature review

On the relative value of imbalanced learning for code smell detection

Contact Info

Product

Resources

About