Natural Language Processing Advancements By Deep Learning: A Survey

Torfi, Amirsina; Shirvani, Rouzbeh A.; Keneshloo, Yaser; Tavaf, Nader; Fox, Edward A.

doi:10.48550/arxiv.2003.01200

Cited by 54 publications

(59 citation statements)

References 121 publications

(146 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Deep learning is nowadays an established way of designing powerful models that are able to effectively solve problems in a wide variety of fields, from natural language processing [1], to computer vision [2] and remote sensing [3]. The most striking successes, such as surpassing human performance on image classification, are due to supervised learning, where huge annotated datasets are used to learn end-to-end models addressing a specific problem.…”

Section: Introductionmentioning

confidence: 99%

Semi-supervised learning for joint SAR and multispectral land cover classification

Montanaro¹,

Valsesia²,

Fracastoro³

et al. 2021

Preprint

View full text Add to dashboard Cite

Self-supervised learning techniques are gaining popularity due to their capability of building models that are effective, even when scarce amounts of labeled data are available. In this paper, we present a framework and specific tasks for self-supervised training of multichannel models, such as the fusion of multispectral and synthetic aperture radar images. We show that the proposed self-supervised approach is highly effective at learning features that correlate with the labels for land cover classification. This is enabled by an explicit design of pretraining tasks which promotes bridging the gaps between sensing modalities and exploiting the spectral characteristics of the input. When limited labels are available, using the proposed self-supervised pretraining and supervised finetuning for land cover classification with SAR and multispectral data outperforms conventional approaches such as purely supervised learning, initialization from training on Imagenet and recent selfsupervised approaches for computer vision tasks.

show abstract

Section: Introductionmentioning

confidence: 99%

Semi-supervised learning for joint SAR and multispectral land cover classification

Montanaro¹,

Valsesia²,

Fracastoro³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…BERT's effectiveness in addressing general NLP tasks with common textual corpora, as compared to traditional machine learning methods for classification, is well supported (González-Carvajal and Garrido-Merchán, 2020). Other surveys and extant research have reviewed NLP tools and industry applications (Kalyanathaya et al, 2019), NLP attention mechanisms (Hu, 2019), NLP for opinion classification (Othman et al, 2015), and deep learning contributions to NLP applications, tasks and objectives (Torfi et al, 2020).…”

Section: Overview Of Methods For Nlp Tasks and Text-to-number Approachesmentioning

confidence: 99%

Textual Data Distributions: Kullback Leibler Textual Distributions Contrasts on GPT-2 Generated Texts, with Supervised, Unsupervised Learning on Vaccine & Market Topics & Sentiment

Samuel,

Palle,

Soares

2021

Preprint

View full text Add to dashboard Cite

Efficient textual data distributions (TDD) alignment and generation are open research problems in textual analytics and NLP. It is presently difficult to parsimoniously and methodologically confirm that two or more natural language datasets belong to similar distributions, and to identify the extent to which textual data possess alignment. This study focuses on addressing a segment of the broader problem described above by applying multiple supervised and unsupervised machine learning (ML) methods to explore the behaviour of TDD by (i) topical alignment, and (ii) by sentiment alignment. Furthermore we use multiple text generation methods including finetuned GPT-2, to generate text by topic and by sentiment. Finally we develop a unique process driven variation of Kullback-Leibler divergence (KLD) application to TDD, named "KL Textual Distributions Contrasts" (KL-TDC) to identify the alignment of machine generated textual corpora with naturally occurring textual corpora. This study thus identifies a unique approach for generating and validating TDD by topic and sentiment, which can be used to help address sparse data problems and other research, practice and classroom situations in need of artificially generated topic or sentiment aligned textual data.

show abstract

“…Efficient representation of sequential data is a long-standing research problem in the NLP domain [34]. The intention is to represent the text data, which is inherently sequential in nature, into a continuous vector representation.…”

Section: A News Encodermentioning

confidence: 99%

DEAP-FAKED: Knowledge Graph based Approach for Fake News Detection

Mayank¹,

Sharma²,

Sharma³

2021

Preprint

View full text Add to dashboard Cite

Fake News on social media platforms has attracted a lot of attention in recent times, primarily for events related to politics (2016 US Presidential elections), healthcare (infodemic during COVID-19), to name a few. Various methods have been proposed for detecting Fake News. The approaches span from exploiting techniques related to network analysis, Natural Language Processing (NLP), and the usage of Graph Neural Networks (GNNs). In this work, we propose DEAP-FAKED, a knowleDgE grAPh FAKe nEws Detection framework for identifying Fake News. Our approach is a combination of the NLP -where we encode the news content, and the GNN technique -where we encode the Knowledge Graph (KG). A variety of these encodings provides a complementary advantage to our detector. We evaluate our framework using two publicly available datasets containing articles from domains such as politics, business, technology, and healthcare. As part of dataset pre-processing, we also remove the bias, such as the source of the articles, which could impact the performance of the models. DEAP-FAKED obtains an F1-score of 88% and 78% for the two datasets, which is an improvement of ∼21%, and ∼3% respectively, which shows the effectiveness of the approach.

show abstract

Natural Language Processing Advancements By Deep Learning: A Survey

Cited by 54 publications

References 121 publications

Semi-supervised learning for joint SAR and multispectral land cover classification

Semi-supervised learning for joint SAR and multispectral land cover classification

Textual Data Distributions: Kullback Leibler Textual Distributions Contrasts on GPT-2 Generated Texts, with Supervised, Unsupervised Learning on Vaccine & Market Topics & Sentiment

DEAP-FAKED: Knowledge Graph based Approach for Fake News Detection

Contact Info

Product

Resources

About