Recent Trends in Deep Learning Based Natural Language Processing [Review Article]

Young, Tom; Hazarika, Devamanyu; Poria, Soujanya; Wang, Zhaoxia

doi:10.1109/mci.2018.2840738

Cited by 2,809 publications

(1,483 citation statements)

References 164 publications

(256 reference statements)

Supporting

Mentioning

1,438

Contrasting

Unclassified

Order By: Relevance

“…Another factor we use is the attention mechanism. In recent years, attention has emerged out as a widely used and important tool in field of deep learning . Attention can be defined as a vector derived as output of dense layer of network using the softmax function.…”

Section: Our Methodsmentioning

confidence: 99%

Fake news detection using deep learning models: A novel approach

Kumar

Asthana

Upadhyay

et al. 2019

Trans Emerging Tel Tech

136

View full text Add to dashboard Cite

With the ever increase in social media usage, it has become necessary to combat the spread of false information and decrease the reliance of information retrieval from such sources. Social platforms are under constant pressure to come up with efficient methods to solve this problem because users' interaction with fake and unreliable news leads to its spread at an individual level. This spreading of misinformation adversely affects the perception about an important activity, and as such, it needs to be dealt with using a modern approach. In this paper, we collect 1356 news instances from various users via Twitter and media sources such as PolitiFact and create several datasets for the real and the fake news stories. Our study compares multiple state‐of‐the‐art approaches such as convolutional neural networks (CNNs), long short‐term memories (LSTMs), ensemble methods, and attention mechanisms. We conclude that CNN + bidirectional LSTM ensembled network with attention mechanism achieved the highest accuracy of 88.78%, whereas Ko et al tackled the fake news identification problem and achieved a detection rate of 85%.

show abstract

Section: Our Methodsmentioning

confidence: 99%

Fake news detection using deep learning models: A novel approach

Kumar

Asthana

Upadhyay

et al. 2019

Trans Emerging Tel Tech

136

View full text Add to dashboard Cite

show abstract

“…Once again, machine learning algorithms could help – this time by facilitating the extraction and structuring of useful clinical information from unstructured EHR data as part of a natural language processing (NLP) system or image recognition task, a task we refer to as ‘information extraction’. In fact, modern deep learning methods are revolutionizing the way computers analyze human language, producing impressive results on a variety of NLP tasks in many domains (26). …”

Section: Setting the Stage: The Two Dimensions Of A Health Outcomementioning

confidence: 99%

“…For this NLP task, the authors represented the text in each document using the classic ‘bag of words’ technique to transform the document into a structured, machine-readable format, which was then used as input to either a logistic regression model or support vector machine. This process of extracting terms from unstructured text to create features or inputs for shallow machine learning models such as support vector machines or logistic regression is a popular approach that has been used for decades to tackle NLP problems (26, 33). When the authors assessed the performance of their algorithms on test sets from four different medical sites, they found that the support vector machine (trained using a cost-sensitive classification approach (34) to maximize sensitivity) consistently achieved the best discrimination, with an area under the receiver operating characteristic curve (AUC) ranging from 0.95 to 0.98 across the four sites.…”

Section: The Four Scenariosmentioning

confidence: 99%

“…Although the Word2Vec inversion approach did not perform as well as the other more traditional machine learning-based NLP approaches also used in this study (described in the previous paragraph), the authors hypothesized that the performance of this newer approach would improve with more training data. In fact, the use of word embeddings as features in machine learning classifiers is a growing trend in the field of NLP (26) and has been shown to produce state-of-the-art results in other medical (40) and non-medical (42) NLP tasks. Indeed, Gehrmann et al (33) recently used word embeddings trained by Word2Vec as inputs to deep learning convolutional neural networks to identify 10 different phenotypes from discharge summaries, including advanced lung disease, metastatic cancer, psychiatric disorders, and chronic neurologic dystrophies, among others.…”

Section: The Four Scenariosmentioning

confidence: 99%

See 1 more Smart Citation

Using Machine Learning to Identify Health Outcomes from Electronic Health Record Data

et al. 2018

View full text Add to dashboard Cite

Purpose of review: Electronic health records (EHRs) contain valuable data for identifying health outcomes, but these data also present numerous challenges when creating computable phenotyping algorithms. Machine learning methods could help with some of these challenges. In this review, we discuss four common scenarios that researchers may find helpful for thinking critically about when and for what tasks machine learning may be used to identify health outcomes from EHR data. Recent findings: We first consider the conditions in which machine learning may be especially useful with respect to two dimensions of a health outcome: 1) the characteristics of its diagnostic criteria, and 2) the format in which its diagnostic data are usually stored within EHR systems. In the first dimension, we propose that for health outcomes with diagnostic criteria involving many clinical factors, vague definitions, or subjective interpretations, machine learning may be useful for modeling the complex diagnostic decision-making process from a vector of clinical inputs to identify individuals with the health outcome. In the second dimension, we propose that for health outcomes where diagnostic information is largely stored in unstructured formats such as free text or images, machine learning may be useful for extracting and structuring this information as part of a natural language processing system or an image recognition task. We then consider these two dimensions jointly to define four common scenarios of health outcomes. For each scenario, we discuss the potential uses for machine learning – first assuming accurate and complete EHR data and then relaxing these assumptions to accommodate the limitations of real-world EHR systems. We illustrate these four scenarios using concrete examples and describe how recent studies have used machine learning to identify these health outcomes from EHR data. Summary: Machine learning has great potential to improve the accuracy and efficiency of health outcome identification from EHR systems, especially under certain conditions. To promote the use of machine learning in EHR-based phenotyping tasks, future work should prioritize efforts to increase the transportability of machine learning algorithms for use in multi-site settings.

show abstract

“…For some health-related topics, there also exists the unbalanced class distribution issues (certain classes are much more frequent than other classes), which can further erode the performance of NLP models [10,13]. To improve the performance on health-related Twitter datasets, substantial time and effort on feature engineering [17,18] is needed for the conventional machine-learning algorithms, including Support Vector Machines (SVM), K-nearest Neighbors(KNN), etc.…”

Section: Introductionmentioning

confidence: 99%

Public Perception Analysis of Tweets During the 2015 Measles Outbreak: Comparative Study Using Convolutional Neural Network Models (Preprint)

Du¹,

Tang²,

Xiang³

et al. 2017

Preprint

View full text Add to dashboard Cite

Background: Timely understanding of public perceptions allows public health agencies to provide up-to-date responses to health crises such as infectious diseases outbreaks.Social media such as Twitter provides an unprecedented way for the prompt assessment of the large-scale public response.Objective: This study has two aims: 1) to develop a scheme for a comprehensive public perception analysis of a measles outbreak from Twitter data; 2) to demonstrate the superiority of the neural network-based approach (compared with conventional machinelearning methods) on measles outbreak-related tweets classification tasks with a relatively small and highly unbalanced gold standard training set. Methods Conclusions:Compared to conventional machine-learning methods, our CNN models showed superiority on measles related tweet classification tasks with a relatively small and highly unbalanced gold standard. Our proposed scheme and neural network based tweets classification system could be used for the analysis of tweets about other infectious diseases such as influenza and Ebola.

show abstract

Recent Trends in Deep Learning Based Natural Language Processing [Review Article]

Cited by 2,809 publications

References 164 publications

Fake news detection using deep learning models: A novel approach

Fake news detection using deep learning models: A novel approach

Using Machine Learning to Identify Health Outcomes from Electronic Health Record Data

Public Perception Analysis of Tweets During the 2015 Measles Outbreak: Comparative Study Using Convolutional Neural Network Models (Preprint)

Contact Info

Product

Resources

About