Review on Word2Vec Word Embedding Neural Net

Sivakumar, S.; Videla, Lakshmi Sarvani; Kumar, Tarun; Nagaraj, J.; Itnal, Shilpa; Haritha, D.

doi:10.1109/icosec49089.2020.9215319

Cited by 40 publications

(14 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Word2Vec is a NLP system that uses neural networks in order to create a distributed word representations in a corpus ( Sivakumar et al, 2020 ). Word2Vec embeddings module was implemented in the Python library Gensim ( ) to train word vectors of the pre-processed text.…”

Section: Methodsmentioning

confidence: 99%

“In the light of evolution:” keratins as exceptional tumor biomarkers

Takan

Karakülah

Louka

et al. 2023

PeerJ

View full text Add to dashboard Cite

Keratins (KRTs) are the intermediate filament-forming proteins of epithelial cells, classified, according to their physicochemical properties, into “soft” and “hard” keratins. They have a key role in several aspects of cancer pathophysiology, including cancer cell invasion and metastasis, and several members of the KRT family serve as diagnostic or prognostic markers. The human genome contains both, functional KRT genes and non-functional KRT pseudogenes, arranged in two uninterrupted clusters on chromosomes 12 and 17. This characteristic renders KRTs ideal for evolutionary studies. Herein, comprehensive phylogenetic analyses of KRT homologous proteins in the genomes of major taxonomic divisions were performed, so as to fill a gap in knowledge regarding the functional implications of keratins in cancer biology among tumor-bearing species. The differential expression profiles of KRTs in diverse types of cancers were investigated by analyzing high-throughput data, as well. Several KRT genes, including the phylogenetically conserved ones, were found to be deregulated across several cancer types and to participate in a common protein-protein interaction network. This indicates that, at least in cancer-bearing species, these genes might have been under similar evolutionary pressure, perhaps to support the same important function(s). In addition, semantic relations between KRTs and cancer were detected through extensive text mining. Therefore, by applying an integrative in silico pipeline, the evolutionary history of KRTs was reconstructed in the context of cancer, and the potential of using non-mammalian species as model organisms in functional studies on human cancer-associated KRT genes was uncovered.

show abstract

Section: Methodsmentioning

confidence: 99%

“In the light of evolution:” keratins as exceptional tumor biomarkers

Takan

Karakülah

Louka

et al. 2023

PeerJ

View full text Add to dashboard Cite

show abstract

“…To construct word vectors, we use the word2vec method in our research. To minimize the complexity of the process, word2vec uses two language models, The continuous bag of words (CBOW) and skip-gram, to learn distributed word representations (Sivakumar et al , 2020). The CBOW model's mechanism is to predict the core word's probability of occurrence based on the contextual words.…”

Section: Methodsmentioning

confidence: 99%

An explainable attention-based bidirectional GRU model for pedagogical classification of MOOCs

Sebbaq

Faddouli

2022

ITSE

View full text Add to dashboard Cite

Purpose The purpose of this study is, First, to leverage the limitation of annotated data and to identify the cognitive level of learning objectives efficiently, this study adopts transfer learning by using word2vec and a bidirectional gated recurrent units (GRU) that can fully take into account the context and improves the classification of the model. This study adds a layer based on attention mechanism (AM), which captures the context vector and gives keywords higher weight for text classification. Second, this study explains the authors’ model’s results with local interpretable model-agnostic explanations (LIME). Design/methodology/approach Bloom's taxonomy levels of cognition are commonly used as a reference standard for identifying e-learning contents. Many action verbs in Bloom's taxonomy, however, overlap at different levels of the hierarchy, causing uncertainty regarding the cognitive level expected. Some studies have looked into the cognitive classification of e-learning content but none has looked into learning objectives. On the other hand, most of these research papers just adopt classical machine learning algorithms. The main constraint of this study is the availability of annotated learning objectives data sets. This study managed to build a data set of 2,400 learning objectives, but this size remains limited. Findings This study’s experiments show that the proposed model achieves highest scores of accuracy: 90.62%, F1-score and loss. The proposed model succeeds in classifying learning objectives, which contain ambiguous verb from the Bloom’s taxonomy action verbs, while the same model without the attention layer fails. This study’s LIME explainer aids in visualizing the most essential features of the text, which contributes to justifying the final classification. Originality/value In this study, the main objective is to propose a model that outperforms the baseline models for learning objectives classification based on the six cognitive levels of Bloom's taxonomy. In this sense, this study builds the bidirectional GRU (BiGRU)-attention model based on the combination of the BiGRU algorithm with the AM. This study feeds the architecture with word2vec embeddings. To prove the effectiveness of the proposed model, this study compares it with four classical machine learning algorithms that are widely used for the cognitive classification of text: Bayes naive, logistic regression, support vector machine and K-nearest neighbors and with GRU. The main constraint related to this study is the absence of annotated data; there is no annotated learning objective data set based on Bloom’s taxonomy's cognitive levels. To overcome this problem, this study seemed to have no choice but to build the data set.

show abstract

“…It is used in different applications, such as sentiment classification, named-entity recognition (NER), POS-tagging, and document analysis. Word2Vec has two techniques: skip gram (SG) and continuous bag of words (CBOW) [64]. In this study, CBOW was used after conducting exhaust experiments, which concluded that CBOW outperformed the SG.…”

Section: Preparing Data For Word Embedding/text Representationmentioning

confidence: 99%

Arabic Sentiment Analysis Based on Word Embeddings and Deep Learning

Elhassan,

Varone,

Ahmed

et al. 2023

Computers

View full text Add to dashboard Cite

Social media networks have grown exponentially over the last two decades, providing the opportunity for users of the internet to communicate and exchange ideas on a variety of topics. The outcome is that opinion mining plays a crucial role in analyzing user opinions and applying these to guide choices, making it one of the most popular areas of research in the field of natural language processing. Despite the fact that several languages, including English, have been the subjects of several studies, not much has been conducted in the area of the Arabic language. The morphological complexities and various dialects of the language make semantic analysis particularly challenging. Moreover, the lack of accurate pre-processing tools and limited resources are constraining factors. This novel study was motivated by the accomplishments of deep learning algorithms and word embeddings in the field of English sentiment analysis. Extensive experiments were conducted based on supervised machine learning in which word embeddings were exploited to determine the sentiment of Arabic reviews. Three deep learning algorithms, convolutional neural networks (CNNs), long short-term memory (LSTM), and a hybrid CNN-LSTM, were introduced. The models used features learned by word embeddings such as Word2Vec and fastText rather than hand-crafted features. The models were tested using two benchmark Arabic datasets: Hotel Arabic Reviews Dataset (HARD) for hotel reviews and Large-Scale Arabic Book Reviews (LARB) for book reviews, with different setups. Comparative experiments utilized the three models with two-word embeddings and different setups of the datasets. The main novelty of this study is to explore the effectiveness of using various word embeddings and different setups of benchmark datasets relating to balance, imbalance, and binary and multi-classification aspects. Findings showed that the best results were obtained in most cases when applying the fastText word embedding using the HARD 2-imbalance dataset for all three proposed models: CNN, LSTM, and CNN-LSTM. Further, the proposed CNN model outperformed the LSTM and CNN-LSTM models for the benchmark HARD dataset by achieving 94.69%, 94.63%, and 94.54% accuracy with fastText, respectively. Although the worst results were obtained for the LABR 3-imbalance dataset using both Word2Vec and FastText, they still outperformed other researchers’ state-of-the-art outcomes applying the same dataset.

show abstract

Review on Word2Vec Word Embedding Neural Net

Cited by 40 publications

References 18 publications

“In the light of evolution:” keratins as exceptional tumor biomarkers

“In the light of evolution:” keratins as exceptional tumor biomarkers

An explainable attention-based bidirectional GRU model for pedagogical classification of MOOCs

Arabic Sentiment Analysis Based on Word Embeddings and Deep Learning

Contact Info

Product

Resources

About