FastEmbed: Predicting vulnerability exploitation possibility based on ensemble machine learning algorithm

Abstract

In recent years, the number of vulnerabilities discovered and publicly disclosed has shown a sharp upward trend. However, the value of exploitation of vulnerabilities varies for attackers, considering that only a small fraction of vulnerabilities are exploited. Therefore, the realization of quick exclusion of the non-exploitable vulnerabilities and optimal patch prioritization on limited resources has become imperative for organizations. Recent works using machine learning techniques predict exploited vulnerabilities by extracting features from open-source intelligence (OSINT). However, in the face of explosive growth of vulnerability information, there is room for improvement in the application of past methods to multiple threat intelligence. A more general method is needed to deal with various threat intelligence sources. Moreover, in previous methods, traditional text processing methods were used to deal with vulnerability related descriptions, which only grasped the static statistical characteristics but ignored the context and the meaning of the words of the text. To address these challenges, we propose an exploit prediction model, which is based on a combination of fastText and LightGBM algorithm and called fastEmbed. We replicate key portions of the state-of-the-art work of exploit prediction and use them as benchmark models. Our model outperforms the baseline model whether in terms of the generalization ability or the prediction ability without temporal intermixing with an average overall improvement of 6.283% by learning the embedding of vulnerability-related text on extremely imbalanced data sets. Besides, in terms of predicting the exploits in the wild, our model also outperforms the baseline model with an F1 measure of 0.586 on the minority class (33.577% improvement over the work using features from darkweb/deepweb). The results demonstrate that the model can improve the ability to describe the exploitability of vulnerabilities and predict exploits in the wild effectively.

A New Vulnerability Reporting Framework for Software Vulnerability Databases

“…After the security vulnerabilities are published in the NVD database, the official notification is completed. However, the criticism that the definitions in the reports do not adequately express the relevant weakness is emphasized by the experts in the field [16]. With these criticisms, security vulnerability databases have been published by different organizations due to the problems caused by the institutional structures of CVE and NVD.…”

Yazılım Güvenlik Açığı Veri Tabanları

“…SecurityFocus en önemli ve en saygın güvenlik açığı veri tabanlarından biridir. NVD veri tabanındaki tanımlamalara göre SecurityFocus listelerindeki tanımlamalar güvenlik açığının etkisini ve sömürülebilirliğini daha spesifik olarak açıklamaktadır (Fang et al 2020).…”

“…Fang vd. (Fang et al 2020), güvenlik açıklarının sadece küçük bir bölümünün saldırganlar tarafından istismar edildiğini belirtmiştir. Bu nedenle istismar edilemez güvenlik açıkları ile diğerlerinin ayırt edilmesinin sınırlı kaynakların verimli kullanımını sağlayacağını vurgulamaktadır.…”

An Extended Benchmark System of Word Embedding Methods for Vulnerability Detection

“…Additionally, a vulnerability prediction system, which was developed based on ensemble machine learning algorithms, employed the FastText model for the encoding task [7]. Given these points, many word embedding techniques had been implemented for detecting vulnerabilities in software code.…”