Post-release detection of a software vulnerability does not only cost a company money to fix, but also results in loss of reputation and damaging litigation. Techniques to prevent and detect vulnerabilities prior to release, therefore, are valuable. We performed empirical case studies on two large, widely-used open source projects: the Mozilla Firefox web browser and the Red Hat Enterprise Linux kernel. We investigated whether software metrics obtained early in the software development life cycle are discriminative of vulnerable code locations, and can guide actions for an organization to take for improvement of code and development team. We also investigated whether the metrics are predictive of vulnerabilities so that prediction models can prioritize validation and verification efforts. The metrics fall into three categories: complexity, code churn, and developer activity metrics. The results indicate that the metrics are discriminative and predictive of vulnerabilities. The predictive model on the three categories of metrics predicted 70.8% of the known vulnerabilities by selecting only 10.9% of the project's files. Similarly, the model for the Red Hat Enterprise Linux kernel found 68.8% of the known vulnerabilities by selecting only 13.0% of the files.

No abstract

Complexity is often hypothesized to be the enemy of software security. If this hypothesis is true, complexity metrics may be used to predict the locale of security problems and can be used to prioritize inspection and testing efforts. We performed statistical analysis on nine complexity metrics from the JavaScript Engine in the Mozilla application framework to find differences in code metrics between vulnerable and nonvulnerable code and to predict vulnerabilities. Our initial results show that complexity metrics can predict vulnerabilities at a low false positive rate, but at a high false negative rate.

Software complexity is often hypothesized to be the enemy of software security. We performed statistical analysis on nine code complexity metrics from the JavaScript Engine in the Mozilla application framework to investigate if this hypothesis is true. Our initial results show that the nine complexity measures have weak correlation (ρ=0.30 at best) with security problems for Mozilla JavaScript Engine. The study should be replicated on more products with design and code-level metrics. It may be necessary to create new complexity metrics to embody the type of complexity that leads to security problems.

Previous studies have shown that software code attributes, such as lines of source code, and history information, such as the number of code changes and the number of faults in prior releases of software, are useful for predicting where faults will occur. In this study of an industrial software system, we investigate the effectiveness of adding information about calling structure to fault prediction models. The addition of calling structure information to a model based solely on non-calling structure code attributes provided noticeable improvement in prediction accuracy, but only marginally improved the best model based on history and non-calling structure code attributes. The best model based on history and non-calling structure code attributes outperformed the best model based on calling and non-calling structure code attributes.