Enhanced word embeddings using multi-semantic representation through lexical chains

Ruas, Terry; Ferreira, Charles Henrique Porto; Grosky, William I.; França, Fabrício Olivetti de; Medeiros, Débora Maria Rossi de

doi:10.1016/j.ins.2020.04.048

Cited by 22 publications

(22 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The same way word2vec [18] inspired many models in NLP [4,26,25], the excellent performance of BERT [8], a Transformer-based model [28], caused its numerous adaption for language tasks [34,6,29,30]. Domain-specific models build on top of Transformers typically outperform their baselines for related tasks [12].…”

Section: Related Workmentioning

confidence: 99%

Testing the Generalization of Neural Language Models for COVID-19 Misinformation Detection

Wahle,

Ashok,

Ruas

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

A drastic rise in potentially life-threatening misinformation has been a by-product of the COVID-19 pandemic. Computational support to identify false information within the massive body of data on the topic is crucial to prevent harm. Researchers proposed many methods for flagging online misinformation related to COVID-19. However, these methods predominantly target specific content types (e.g., news) or platforms (e.g., Twitter). The methods' capabilities to generalize were largely unclear so far. We evaluate fifteen Transformer-based models on five COVID-19 misinformation datasets that include social media posts, news articles, and scientific papers to fill this gap. We show tokenizers and models tailored to COVID-19 data do not provide a significant advantage over general-purpose ones. Our study provides a realistic assessment of models for detecting COVID-19 misinformation. We expect that evaluating a broad spectrum of datasets and models will benefit future research in developing misinformation detection systems.

show abstract

Section: Related Workmentioning

confidence: 99%

Testing the Generalization of Neural Language Models for COVID-19 Misinformation Detection

Wahle,

Ashok,

Ruas

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Word2vec and doc2vec can both capture latent semantic meaning from textual data using efficient neural network language models. Prediction-based word embedding models, such as word2vec and doc2vec, have proven themselves superior to count-based models, such as BOW, for several problems in Natural Language Processing (NLP), such as quantifying word similarity [15], classifying documents [16], and analyzing sentiment [17]. Gharavi et al employed word embeddings to perform text alignment for sentences [18].…”

Section: Related Workmentioning

confidence: 99%

“…Recently, the NLP community adapted and extended the neural language model BERT [20] for a variety of tasks [21]- [26], similar to the way that word2vec [13] has influenced many later models in NLP [15], [16], [27]. Based on the Transformer architecture [28], BERT employs two pre-training tasks, i.e., Masked Language Model (MLM) and Next Sentence Prediction (NSP), to capture general aspects of language.…”

Section: Related Workmentioning

confidence: 99%

Identifying Machine-Paraphrased Plagiarism

Ruas¹,

Foltýnek²,

Meuschke³

et al. 2021

Self Cite

View full text Add to dashboard Cite

Employing paraphrasing tools to conceal plagiarized text is a severe threat to academic integrity. To enable the detection of machine-paraphrased text, we evaluate the effectiveness of five pre-trained word embedding models combined with machine learning classifiers and state-of-the-art neural language models. We analyze preprints of research papers, graduation theses, and Wikipedia articles, which we paraphrased using different configurations of the tools SpinBot and SpinnerChief. The best performing technique, Longformer, achieved an average F1 score of 80.99% (F1=99.68% for SpinBot and F1=71.64% for SpinnerChief cases), while human evaluators achieved F1=78.4% for SpinBot and F1=65.6% for SpinnerChief cases. We show that the automated classification alleviates shortcomings of widelyused text-matching systems, such as Turnitin and PlagScan. To facilitate future research, all data 1 , code 2 , and two web applications 34 showcasing our contributions are openly available.

show abstract

“…Word embedding is a technique for analyzing the context of words in a sentence and converting them into a vector value. Word embedding methods include GloVe [16], Fasttext [17], and Word2vec [18]. Word2vec fails to consider the cooccurrence frequency of entire sentences because learning is performed only in a user-specified window.…”

Section: B Word2vec-based Word Embeddingmentioning

confidence: 99%

“…This generates a vector of words that are not found in the dictionary; moreover, its training process is fast [17]. Word2vec was proposed by Google after the neural network language model was improved [18]. It infers the meaning of words based on the distribution hypothesis that words in similar locations have a similar meaning.…”

Section: B Word2vec-based Word Embeddingmentioning

confidence: 99%

Word-Embedding-Based Traffic Document Classification Model for Detecting Emerging Risks Using Sentiment Similarity Weight

2020

View full text Add to dashboard Cite

With the increase in traffic accident rates, traffic risk detection is becoming increasingly important. Moreover, it is necessary to provide appropriate traffic information considering user locations and routes and design an analysis method accordingly. This paper proposes a word-embedding-based traffic document classification model for detecting emerging risks using a quantity termed sentiment similarity weight (SSW). The proposed method detects emerging risks by considering and classifying the importance and polarity of keywords in traffic document. Conventional sentiment analysis methods fail to utilize semantically significant keywords unless they are included in a sentiment dictionary. In this study, through word imputation using an established similarity dictionary and by widening the limited utilization range, the proposed method overcomes the disadvantage of sentiment dictionaries. The proposed method is evaluated through three tests. In the first, the similarity between keywords is measured, and thus model accuracy is evaluated. In the second test, three classifiers for emerging risk classification are compared. In the last test, emerging risk detection is assessed according to whether the proposed SSW is applied, and its effectiveness is therefore verified. The evaluation results demonstrate that the proposed traffic-related document classification model using the SSW has an f-measure of 0.907, indicating satisfactory performance. Therefore, the proposed SSW can be effectively used as a parameter in traffic-related document classification and enables the detection of emerging risks.

show abstract

Enhanced word embeddings using multi-semantic representation through lexical chains

Cited by 22 publications

References 34 publications

Testing the Generalization of Neural Language Models for COVID-19 Misinformation Detection

Testing the Generalization of Neural Language Models for COVID-19 Misinformation Detection

Identifying Machine-Paraphrased Plagiarism

Word-Embedding-Based Traffic Document Classification Model for Detecting Emerging Risks Using Sentiment Similarity Weight

Contact Info

Product

Resources

About