Distributed Representation Computation Using CBOW Model and Skip–gram Model

Onishi, Takamune; Shiina, Hiromitsu

doi:10.1109/iiai-aai50415.2020.00179

Cited by 19 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A detailed comparison is presented in Table . 1. Skip-gram performs better with higher scores as stated and compared in [29] under the same vector dimensions and training window size. We also observed that increasing the training epoch increased the performance of CBOW and Skip-gram when we fixed the window size and vector dimension.…”

Section: Fundamentals: Word2vec Models and 5g Protocolsmentioning

confidence: 61%

“…It is considered a landmark work in the field of deep learning (DL) and NLP, introducing Word2Vec, an effective word vector representation that lays the foundation for later word embedding research in [36]. Based on the continuous bag-of-words (CBOW) and Skip-gram, a new network that combines CBOW and Skip-gram together by sharing a weight matrix was developed [29]. It was summarized that Skip-gram has better word representation distribution however, is slower in speed compared to CBOW.…”

Section: Fundamentals: Word2vec Models and 5g Protocolsmentioning

confidence: 99%

See 1 more Smart Citation

DEFT: A Novel Deep Framework for Fuzz Testing Performance Evaluation in NextG Vulnerability Detection

Peng,

Li,

Arya

et al. 2023

IEEE Access

View full text Add to dashboard Cite

As the evolution of telecommunication technology advances rapidly, the next-generation networks have undeniably become integral components of modern society. However, given the vast infrastructure requirement for their deployment and the intricacy of their protocols, they inevitably present considerable security challenges. Recently, fuzz testing has emerged as a prominent method of system assurance, gaining substantial traction in the realms of 5G vulnerability detection and performance assessment. It can test and exploit 5G networks to detect invalid integrity protection and security procedure bypasses. The fuzzed system behaviors contain information that could indicate the system's health and potential vulnerabilities. This paper proposes a novel modeling framework, coined as DEFT, to perform causation analysis in the system under test (SUT) in detecting the fuzzed location, therefore, DEFT can be further applied to identifying the type of attacks or abnormal inputs from the partial system profiling for the impacted behaviors. In particular, we show, for the first time, that by utilizing the DEFT, we can precisely detect the fuzzed layer in the log file which can then be further utilized to identify the root cause of vulnerabilities with high accuracy using only a tiny segment of the log file in real-time. To test the DEFT for real-world application scenarios, we generate an unbiased dataset by performing fuzz testing to trigger potential vulnerabilities and unintended behaviors on our 5G test bed and recording the system behaviors via regular logging information. We then analyze a random segment of the log files to detect the root cause of the vulnerabilities via the processing of the word embedding vectors that are enabled by two architectures, Skipgram and continuous bag-of-words (CBOW), learning the underlying word representations for each word using neural networks. The DEFT can effectively identify the fuzzed location and thereby the occurrence of unknown attacks, therefore, can be considered a crucial step in performing a timely attack response and the root cause analysis. We assess multiple machine learning models integrated with the two architectures by the input segment length of log files, computation complexity, and accuracy matrix. It is shown that, for the Skip-gram, an area under the curve (AUC) value of 0.938 is achieved for the Fully Connected Network (FCN) model while considering only 50% length of the original log file, thereby significantly reducing the computational complexity. By eliminating the necessity for a 100% comprehensive log file and instead utilizing a 50% log file, we have effectuated an 8% decrease in testing duration. We make an important observation that, compared with CBOW, Skip-gram shows superior performance when only a smaller fraction of the log file is considered for vulnerability detection, thereby, can readily be applied to real-time 5G applications. Our results provide a valuable direction in vulnerability detection and enable timely attack response.

show abstract

Section: Fundamentals: Word2vec Models and 5g Protocolsmentioning

confidence: 61%

Section: Fundamentals: Word2vec Models and 5g Protocolsmentioning

confidence: 99%

DEFT: A Novel Deep Framework for Fuzz Testing Performance Evaluation in NextG Vulnerability Detection

Peng,

Li,

Arya

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Irsoy et al [11] argues that CBOW can perform equally well as Skipgram when a bug in the CBOW gradient update is fixed. Onishi et al [12] proposes a method to combine both models to achieve faster learning speed and a more accurate distribution representation of words. Xiong et al [13] suggests new models based on optimization and regression methods to enhance the performance of both CBOW and Skip-gram.…”

Section: Related Workmentioning

confidence: 99%

The comparison and analysis of Skip-gram and CBOW in creating financial sentimental dictionary

Zhang,

Zhang

2024

ACE

View full text Add to dashboard Cite

Textual analysis is increasingly used in various fields due to data availability, computing power, and machine learning techniques. In finance, sentiment analysis is essential for obtaining excess returns, and building domain-specific lexicons using word2vec is a prevalent method. The CBOW and Skip-gram algorithms have different predictive methodologies and performances depending on the task and dataset. This paper reviews financial sentiment analysis using a dictionary method and compares the performance of the two algorithms. CBOW trains faster than Skip-gram when dealing with a small amount of text data, but as the amount of data increases, Skip-gram becomes more efficient. Besides, the Skip-gram captures more synonyms of the selected words than CBOW.

show abstract

“…The two approaches used by Word2vec are Continuous Bag of Words (CBOW) and Skip-gram [33]. A vector-distributed numerical representation of word characteristics is produced by Word2vec.…”

Section: Word Embedding: Word2vecmentioning

confidence: 99%

Depression Detection Model Usingword and Sentence Embedding With Different Classifiers

Rai,

Goyal,

Choudhary

2023

IJEAST

View full text Add to dashboard Cite

Determining depression is hugely important because it affects people's wellness. The common practice of diagnosing depression usually involves a physical session with a practitioner, which reduces the possibility of finding patients who are depressed. Instead, they investigate the possibility of detecting depression solely from textual data based on the posts people post on social media profiles. This research work consists of how Reddit users' text messages can be processed to perform a classification task to identify depression. In order to detect depression, we used Word and Sentence Embeddings. Different embeddings are tested with algorithms such as SVM classifiers, KNN classifiers, and decision tree classifiers, and the results are compared to those from various datasets. Evaluation metrics such as accuracy and precision are used to evaluate the performance of classifiers. USE embedding outperformed Word embeddings with a maximum accuracy of 92%, precision of 95%, recall of 96%, and F1 score of 92%. Results from the suggested method are compared to those from other methods and are found to be satisfactory.

show abstract

Distributed Representation Computation Using CBOW Model and Skip–gram Model

Cited by 19 publications

References 0 publications

DEFT: A Novel Deep Framework for Fuzz Testing Performance Evaluation in NextG Vulnerability Detection

DEFT: A Novel Deep Framework for Fuzz Testing Performance Evaluation in NextG Vulnerability Detection

The comparison and analysis of Skip-gram and CBOW in creating financial sentimental dictionary

Depression Detection Model Usingword and Sentence Embedding With Different Classifiers

Contact Info

Product

Resources

About