Identification of Cybersecurity Specific Content Using the Doc2Vec Language Model

Quantifying the Significance and Relevance of Cyber-Security Text Through Textual Similarity and Cyber-Security Knowledge Graph

“…In our earlier work [1] we proposed a system to identify threat information from publicly available information sources. With some modification to the original design, the proposed system architecture would be as shown in Fig.…”

“…The Natural Language Filter module is a language model that is trained to identify and filter the security-related text documents from publicly available information sources. In [1] we experimented with the Doc2Vec language model to utilize as Natural Language Filter by training it with over 1 million security-specific text documents. The model would compare the cosine similarity of the vector representation of any incoming text document with its training document and filter out the documents that have less than 70% similarity.…”

“…Even though there are approaches that automatically share information between machines through structured information sharing such as Structured Threat Information Expression (STIX) 1 and its corresponding protocol Trusted Automated Exchange of Intelligence Information (TAXII), the need to process unstructured text reports that might be shared via email or forums still exists. For example, dark-web forums provide valuable threat information, if the noise can be segregated, with less effort.…”

“…In our earlier work [1], we proposed the overall architecture of this autonomous system to identify user-specific threat information from publicly available information sources. In that work, we proposed to filter the cyber-security specific content from the publicly available text information.…”

“…The objective relevance measures how well the topic of the information matches the domain, and subjective relevance deals with user-specific situations. In [1] we attempted to identify the text documents that have objective relevance within the domain of cyber-security. In this work our goal is to seek a way to quantify the subjective relevance of the text documents alongside with its potential significance, which can be customized to meet user-specific needs by utilizing existing Natural Language Processing (NLP) techniques and tools.…”