Evaluating domain-specific metric thresholds

Mori, Allan; Vale, Gustavo; Viggiato, Markos; Oliveira, Johnatan; Figueiredo, Eduardo; Cirilo, Elder; Jamshidi, Pooyan; Kästner, Christian

doi:10.1145/3194164.3194173

Cited by 25 publications

(15 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors emphasised the need to pay a special attention to software categories when comparing systems in distinct categories with predefined thresholds already available in the literature. For example, empirical results from the study revealed that out of ten distinct categories selected (including Audio and Video, Graphics, Recently, Mori et al [67] argued that while deriving reliable thresholds for software metrics has been a ongoing research concern, there is still a lack of evidence about threshold variation across different software domains. In an attempt to address this limitation, the authors whether and how thresholds vary across domains in a study on 3,107 software systems from 15 domains In general, earlier studies have shown that application domains indeed have an effect on software metric values and thresholds.…”

Section: Related Workmentioning

confidence: 99%

Detecting Java software similarities by using different clustering techniques

Capiluppi

Ruscio

Rocco

et al. 2020

Information and Software Technology

View full text Add to dashboard Cite

Background-Research on empirical software engineering has increasingly been conducted by analysing and measuring vast amounts of software systems. Hundreds, thousands and even millions of systems have been (and are) considered by researchers, and often within the same study, in order to test theories, demonstrate approaches or run prediction models. A much less investigated aspect is whether the collected metrics might be context-specific, or whether systems should be better analysed in clusters. Objective-The objectives of this study are (i) to define a set of clustering techniques that might be used to group similar software systems, and (ii) to evaluate whether a suite of well-known object-oriented metrics is context-specific, and its values differ along the defined clusters. Method-We group software systems based on three different clustering techniques, and we collect the values of the metrics suite in each cluster. We then test whether clusters are statistically different between each other, using the Kolgomorov-Smirnov (KS) hypothesis testing. Results-Our results show that, for two of the used techniques, the KS null hypothesis (e.g., the clusters come from the same population) is rejected for

show abstract

Section: Related Workmentioning

confidence: 99%

Detecting Java software similarities by using different clustering techniques

Capiluppi

Ruscio

Rocco

et al. 2020

Information and Software Technology

View full text Add to dashboard Cite

show abstract

“…Even more, the results show that software metric values follow the same probability distribution regardless of the application domain. Mori et al [28] also analyzed the impact of domains on derived thresholds. In contrast to the study described above, they found evidence that software metrics thresholds are sensitive to the software domain, but we can still find domains that have similar thresholds for some of the analyzed software metrics [28].…”

Section: Studymentioning

confidence: 99%

“…Number of software projects Alves et al [1] 100 Arar and Ayan [2] 10 Ferreira et al [10] 40 Filo et al [12] 111 (from Qualitas Corpus [40]) Fontana et al [13] 74 (from Qualitas Corpus [40]) Mori et al [28] 3,107 Oliveira et al [30] 106 (from Qualitas Corpus [40]) Yamashita et al [44] 4,780 Vale et al [42] 103 (from Qualitas Corpus [40]) [10] and Arar and Ayan [2] used a smaller benchmark, including 40 and 10 projects, respectively.…”

Section: Studymentioning

confidence: 99%

Comparison of systematically derived software metrics thresholds for object-oriented programming languages

Beranič

Heričko

2020

ComSIS

View full text Add to dashboard Cite

Without reliable software metrics threshold values, the efficient quality evaluation of software could not be done. In order to derive reliable thresholds, we have to address several challenges, which impact the final result. For instance, software metrics implementations vary in various software metrics tools, including varying threshold values that result from different threshold derivation approaches. In addition, the programming language is also another important aspect. In this paper, we present the results of an empirical study aimed at comparing systematically obtained threshold values for nine software metrics in four object-oriented programming languages (i.e., Java, C++, C#, and Python). We addressed challenges in the threshold derivation domain within introduced adjustments of the benchmarkbased threshold derivation approach. The data set was selected in a uniform way, allowing derivation repeatability, while input values were collected using a single software metric tool, enabling the comparison of derived thresholds among the chosen object-oriented programming languages. Within the performed empirical study, the comparison reveals that threshold values differ between different programming languages.

show abstract

“…The proposed technique was more effective in predicting bad smells than Alves's technique. Mori et al [30 ] have studied a benchmark of large data set composed of >3000 systems coming from 15 different domains. The study derives thresholds by ranking entities and choosing the top ranks (90 and 95%) as candidate thresholds.…”

Section: Related Workmentioning

confidence: 99%

Comparison of threshold identification techniques for object-oriented software metrics

Shatnawi¹

2020

IET Software

View full text Add to dashboard Cite

Quality assurance is a continuous process throughout the project lifecycle from inception till post‐delivery. Software metrics are tools to help developers in achieving software quality objectives. Software metrics are used to predict the fault‐proneness of classes in software using machine‐learning and statistical techniques. However, these methodologies are difficult for daily tasks. Simpler and on the fly methodologies such as threshold values are needed. Metric thresholds can be used to control software quality and to recommend improvements on software code. Thresholds detect the parts of software that need more verification and validation. Many threshold identification techniques were proposed in previous research. However, the techniques do not provide consistent thresholds. The authors compare eight threshold identification techniques to diagnose software fault‐proneness. The eight techniques are derived from diagnosis measures such as specificity, sensitivity, recall and precision. Five threshold identification techniques have derived thresholds that are skewed and have large standard deviations. Only three techniques are selected for threshold identification based on consistency and variation in selecting thresholds of software metrics in the systems under study. These techniques find thresholds that have the least variation among the studied techniques. The median of the 11 systems is selected as a representative of all thresholds.

show abstract

Evaluating domain-specific metric thresholds

Cited by 25 publications

References 24 publications

Detecting Java software similarities by using different clustering techniques

Detecting Java software similarities by using different clustering techniques

Comparison of systematically derived software metrics thresholds for object-oriented programming languages

Comparison of threshold identification techniques for object-oriented software metrics

Contact Info

Product

Resources

About