Empirical analysis of the relationship between CC and SLOC in a large corpus of Java methods and C functions

Landman, Davy; Serebrenik, Alexander; Bouwers, Eric; Vinju, Jurgen

doi:10.1002/smr.1760

Cited by 24 publications

(20 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…State of the art: Source code analysis has its firm fundaments in compiler (frontend) construction [2] and reverse engineering [20]. Based on this theory and [8,15]. However, mining information from source code at scale, made available in an integrated platform, including the acquisition, extraction, and querying of source code from groups of arbitrarily selected projects is just beyond the current state-of-the-art.…”

Section: Development Of Source Code Analysis Toolsmentioning

confidence: 99%

Developer-Centric Knowledge Mining from Large Open-Source Software Repositories (CROSSMINER)

Bagnato

Barmpis

Bessis

et al. 2018

Software Technologies: Applications and Foundations

View full text Add to dashboard Cite

Deciding if an OSS project meets the required standards for adoption is hard, and keeping up-to-date with a rapidly evolving project is even harder. Making decisions about quality and adoption involves analysing code, documentation, online discussions, and issue trackers. There is too much information to process manually and it is common that uninformed decisions have to be made with detrimental effects.

show abstract

Section: Development Of Source Code Analysis Toolsmentioning

confidence: 99%

Developer-Centric Knowledge Mining from Large Open-Source Software Repositories (CROSSMINER)

Bagnato

Barmpis

Bessis

et al. 2018

Software Technologies: Applications and Foundations

View full text Add to dashboard Cite

show abstract

“…Many studies have explored correlations between various software metrics such as McCabe's cyclomatic complexity (Landman et al 2016;Henry and Selig 1990;Henry et al 1981;Tashtoush et al 2014;Jay et al 2009;Meulen and Revilla 2007), lines of code (Landman et al 2016;Henry and Selig 1990;Tashtoush et al 2014;Jay et al 2009;Meulen and Revilla 2007), Halstead's metrics (Henry and Selig 1990;Henry et al 1981;Tashtoush et al 2014; Meulen and Revilla 2007) Kafura's information flow (Henry et al 1981), Number of comments, Meulen and Revilla (2007), etc. Most of these studies have observed that code metrics are highly correlated.…”

Section: Introductionmentioning

confidence: 99%

Effects of measurements on correlations of software code metrics

2019

View full text Add to dashboard Cite

Context Software metrics play a significant role in many areas in the life-cycle of software including forecasting defects and foretelling stories regarding maintenance, cost, etc. through predictive analysis. Many studies have found code metrics correlated to each other at such a high level that such correlated code metrics are considered redundant, which implies it is enough to keep track of a single metric from a list of highly correlated metrics. Objective Software is developed incrementally over a period. Traditionally, code metrics are measured cumulatively as cumulative sum or running sum. When a code metric is measured based on the values from individual revisions or commits without consolidating values from past revisions, indicating the natural development of software, this study identifies such a type of measure as organic. Density and average are two other ways of measuring metrics. This empirical study focuses on whether measurement types influence correlations of code metrics. Method To investigate the objective, this empirical study has collected 24 code metrics classified into four categories, according to the measurement types of the metrics, from 11,874 software revisions (i.e., commits) of 21 open source projects from eight well-known organizations. Kendall's τ-B is used for computing correlations. To determine whether there is a significant difference between cumulative and organic metrics, Mann-Whitney U test, Wilcoxon signed rank test, and paired-samples sign test are performed. Results The cumulative metrics are found to be highly correlated to each other with an average coefficient of 0.79. For corresponding organic metrics, it is 0.49. When individual correlation coefficients between these two measure types are compared, correlations between organic metrics are found to be significantly lower (with p <0.01) than cumulative metrics. Our results indicate that the cumulative nature of metrics makes them highly correlated, implying cumulative measurement is a major source of collinearity between cumulative metrics. Another interesting observation is that correlations between metrics from different categories are weak.

show abstract

“…These results are somewhat refuted by the analysis of millions of Java and C functions conducted by Landman and colleagues [34]; the authors conclude that the relationship between Cyclomatic Complexity and LOC is not that strong and argue in favor of using the two metrics.…”

Section: Chapter 2 Related Workmentioning

confidence: 93%

“…We also note that there is an ongoing, unresolved debate about the usefulness of the Cyclomatic Complexity metric. While numerous research studies (e.g., [43], [44], [45], [48], [50]) have argued in favor of the existence of a strong correlation between Cyclomatic Complexity and Lines of Code, and therefore advocate for the use of only one of them, namely LOC because it is simpler to compute, some authors on the contrary claim that the correlation between CC and LOC is only moderate when the source code is analyzed in modules rather than as an aggregate unit (larger program) [49], [51] as claimed by the former. Though our intent is not to settle the ongoing debate, we, however, investigate these claims for application code and test code at the method level (lowest level of granularity for complexity metrics) and as an aggregate…”

Section: Chapter 2 Related Workmentioning

confidence: 99%

“…One critical challenge that numerous studies and hypothesis [47], [48], [49], [50], [51] have not explored on correlation of metrics is the factors that affect the correlation being: the kind of code, the kind of software and the kind of metric. This is evident in the large-scale software study conducted by Landman and colleagues [49], the work of Maine While we do agree that it really makes sense to transform non-normal data to a normal one, we are skeptical (disagree) with the results it produces as this can be very misleading [51], [52]. In addition, on the issue of normality, Landman presents a series of results gained from a transformed non-normal distribution, using parametric tests on the transformed data, which though this is a procedure accepted in the field, seems inappropriate and misleading.…”

Section: Chapter 1 Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Empirical Study Investigating the Predictors of Software Metric Correlation in Application Code and Test Code.

Afriyie¹

View full text Add to dashboard Cite

On non-trivial software, the large test code base needs adequate maintenance similarly to the application code. It has often been argued that test code should be simple, some authors even arguing that a test script should be reduced to a single control flow. If this is indeed what happens in practice, then we believe test code maintenance will be very different from application code maintenance, despite the fact that both are source code. Using a number of large open source software, we compare application code and test code by using a series of well-known source code complexity metrics. The results reveal that application code is, as expected, more complex than test code but not necessarily so much more complex. The study also confirms the assertion that test code is not as simple as it should be, at least as advocated in textbooks, and may therefore be very complex to maintain. Using complexity metrics to determine the difference, our findings also reveal that, the kind of code determines the extent of monotonicity between the number of lines of code (LOC) and cyclomatic complexity (CC). While a number of authors hypothesize and experimentally confirm that CC has a very strong correlation with LOC, justifying the use of LOC in place of CC (and Halstead Effort), we believe that this strong correlation is prevalent only in production code as results from test code prove otherwise. In test code, there exist a very weak (or almost no) correlation between (a) LOC and CC, (b) Halstead Effort and CC, and (c) LOC and Halstead Effort. We therefore argue that the level of correlation depends on at least three factors namely: the kind of code, the kind of software and the kind of metric. Therefore, we believe it would be inappropriate, without considering these factors to substitute one metric with another using some generalized correlation hypothesis. The results gained thus far contradict the notion that some metrics are correlated (that is if the right factors are not considered). Given the weak monotonicity between CC and LOC, CC and Halstead Effort for test code, we disagree that CC and Halstead metrics are redundant as some studies suggest. Rather, we advocate for the use of CC over LOC (or both or cylomatic density) due to numerous advantages CC has over LOC such as the fact that CC is perceived to better reflect cognitive complexity, numerical complexity, adequacy of testing, interdependency and code refactoring that cannot be accounted for simply by LOC.

show abstract

Empirical analysis of the relationship between CC and SLOC in a large corpus of Java methods and C functions

Cited by 24 publications

References 72 publications

Developer-Centric Knowledge Mining from Large Open-Source Software Repositories (CROSSMINER)

Developer-Centric Knowledge Mining from Large Open-Source Software Repositories (CROSSMINER)

Effects of measurements on correlations of software code metrics

An Empirical Study Investigating the Predictors of Software Metric Correlation in Application Code and Test Code.

Contact Info

Product

Resources

About