Evaluation of Software Product Quality Metrics

Abstract

Computing devices and associated software govern everyday life, and form the backbone of safety critical systems in banking, healthcare, automotive and other fields. Increasing system complexity, quickly evolving technologies and paradigm shifts have kept software quality research at the forefront. Standards such as ISO's 25010 express it in terms of sub-characteristics such as maintainability, reliability and security. A significant body of literature attempts to link these subcharacteristics with software metric values, with the end goal of creating a metricbased model of software product quality. However, research also identifies the most important existing barriers. Among them we mention the diversity of software application types, development platforms and languages. Additionally, unified definitions to make software metrics truly language-agnostic do not exist, and would be difficult to implement given programming language levels of variety. This is compounded by the fact that many existing studies do not detail their methodology and tooling, which precludes researchers from creating surveys to enable data analysis on a larger scale. In our paper, we propose a comprehensive study of metric values in the context of three complex, open-source applications. We align our methodology and tooling with that of existing research, and present it in detail in order to facilitate comparative evaluation. We study metric values during the entire 18-year development history of our target applications, in order to capture the longitudinal view that we found lacking in existing literature. We identify metric dependencies and check their consistency across applications and their versions. At each step, we carry out comparative evaluation with existing research and present our results.

A Statistical Evaluation of The Depth of Inheritance Tree Metric for Open-Source Applications Developed in Java

“…These values are below 0.88. We may explain this result by the fact that in [15], the DIT mean values are given per app, and in our case, per class. Although we note that according to [1], the DIT mean value of TuxGuitar equals to 1.46 and is in the range of acceptable values.…”

“…Parameters of the transformation (12) were estimated by the maximum likelihood method (10) for the function ( 14) with restrictions (13). Estimators for parameters of the transformation (12) and pdf (15) are: , , and…”

“…We have compared these results with the DIT mean values of three popular open-source apps developed in Java: FreeMind, TuxGuitar, and jEdit having over 465k, 131k, and 56k downloads in 2019, respectively [15]. The DIT mean values of FreeMind, TuxGuitar, and jEdit equal to 0.79, 0.42, and 0.87, respectively.…”

“…The Depth of Inheritance Tree (DIT) metric was firstly defined by Chidamber and Kemerer in [5]. The DIT metric, along with others, is used for estimating the faults [7], [19], [21], reusability [13] and other quality indicators [15] in software systems, including open-source ones [2], [8], [15], [21]. Additionally, the DIT metric is considered as a factor influencing the cost of testing [3], [20].…”

“…However, there are apps in Java, in which the DIT values for some classes are significantly greater than 5. For example, in the well-known jEdit app version 5.5.0, the software metrics of which are analyzed in [15], out of 1390 classes, 179 ones have DIT values greater than 5. In this app, there are 32 classes that DIT values are more than twice the maximum recommended DIT value of 5, and the DIT values for the two classes are more than 10 times 5: 52 and 53 for the org.gjt.sp.jedit.pluginmgr.InstallPanel and org.gjt.sp.jedit.pluginmgr.ManagePanel respectively.…”