Toward Non-security Failures as a Predictor of Security Faults and Failures

Abstract

Abstract. In the search for metrics that can predict the presence of vulnerabilities early in the software life cycle, there may be some benefit to choosing metrics from the non-security realm. We analyzed non-security and security failure data reported for the year 2007 of a Cisco software system. We used non-security failure reports as input variables into a classification and regression tree (CART) model to determine the probability that a component will have at least one vulnerability. Using CART, we ranked all of the system components in descending order of their probabilities and found that 57% of the vulnerable components were in the top nine percent of the total component ranking, but with a 48% false positive rate. The results indicate that nonsecurity failures can be used as one of the input variables for security-related prediction models.

Predicting Attack-prone Components

“…The weaker metrics have less significant splits, although the p-values are below 0.05, than the strongly-correlated metric. However, our observations in [12], and in comparison of the results presented here with those in [13], we find that weaklycorrelated metrics can lead to more splits than strongly-correlated metrics, and that many splits can yield good separation between attack-prone and not attack-prone components. …”

“…We do not restrict our metrics to the security realm, however. Our prior work [9,12,13] indicates that non-security failures are positively correlated with security failures, and we therefore do not exclude non-security warnings as candidate metrics.…”

Predicting Vulnerable Software Components via Text Mining

“…Gegick et al [14] investigated whether non-security failure reports could be used to predict whether a given component is vulnerable. In the context of a Cisco software system, the authors found a 0.4 correlation between security faults and non-security failures.…”

Searching for a Needle in a Haystack: Predicting Security Vulnerabilities for Windows Vista

“…Gegick et al used code-level metrics such as lines of code, code churn, and number of static tool alerts [8] as well as past non-security faults [7] to predict security faults. In the most recent work, Gegick et al achieved a precision of 0.52 and a recall of 0.57.…”