An attention-gated convolutional neural network for sentence classification

Liu, Yang; Ji, Lixin; Huang, Ruiyang; Ming, Tuosiyu; Gao, Chao; Zhang, Jianpeng

doi:10.3233/ida-184311

Cited by 37 publications

(15 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the weights and the abstract features are convolved at the same level, the significant features identified by the gating weights is very monotonous. To better discover contextual information in text classification, Yang Liu et al introduced a new CNN model (AGCNN) for sentence classification, which generated the gating weights by a variety of specialized convolution kernels to integrate the contextual information of a particular context window into the control weights [31]. And to achieve better performance with aspect-based sentiment analysis, Wei Xue et al proposed a model based on gated convolutional neural networks, which can selectively output the sentiment features according to the given aspect or entity [32].…”

Section: Related Workmentioning

confidence: 99%

MGL-CNN: A Hierarchical Posts Representations Model for Identifying Depressed Individuals in Online Forums

Rao

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

More users suffering from depression turn to online forums to express their problems and seek help. In such forums, there is often a large volume of posts with sensitive content, indicating that the user has a risk of suicide and self-harm. Early detection of depression using appropriate deep learning models and social media data can prevent potential self-harm. However, existing depression detection models are not powerful enough to capture critical sentiment information from the large volume of posts published by each user, which makes the performance of these models not satisfying. To address this problem, we propose a hierarchical posts representations model named Multi-Gated LeakyReLU CNN (MGL-CNN) for identifying depressed individuals in online forums. The model consists of two parts: the first one is a post-level operation, which is used to learn the representation of each post of the user, and the second one is a user-level operation, which is used to obtain the overall representation of the user's emotional state. Besides, we propose another depression detection model by changing the number of gated units in the MGL-CNN, which is named Single-Gated LeakyReLU CNN (SGL-CNN). We show how to use our models to identify depressed users through a lot of posted content. Experimental results showed that our models performed better than the previous stateof-the-art models on the Reddit Self-reported Depression Diagnosis dataset, and also performed well on the Early Detection of Depression dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

MGL-CNN: A Hierarchical Posts Representations Model for Identifying Depressed Individuals in Online Forums

Rao

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Attention models have also been used with CNN and time series data. The work in [43] proposed an attention gated CNN for sentence classification, and the work in [44] introduced a temporal causal discovery framework (TCDF) for learning causal relationships in time series data. However, many innovative CNN architectures mentioned above have not yet been explored for renewable energy applications.…”

Section: Of 29mentioning

confidence: 99%

“…The works in [65][66][67] are a few examples where attention blocks were proposed and used with LSTM architectures for time series forecasting. Attention blocks have also been used with CNN architectures [40,43,68,69] for image classification and time series data. One of the noteworthy contributions of the attention mechanism for time series forecasting can be found in [70].…”

Section: The Proposed Modelmentioning

confidence: 99%

Multi-Step Short-Term Wind Speed Prediction Using a Residual Dilated Causal Convolutional Network with Nonlinear Attention

Kumar

Tzou

2020

Energies

View full text Add to dashboard Cite

Wind energy is the most used renewable energy worldwide second only to hydropower. However, the stochastic nature of wind speed makes it harder for wind farms to manage the future power production and maintenance schedules efficiently. Many wind speed prediction models exist that focus on advance neural networks and/or preprocessing techniques to improve the accuracy. Since most of these models require a large amount of historic wind data and are validated using the data split method, the application to real-world scenarios cannot be determined. In this paper, we present a multi-step univariate prediction model for wind speed data inspired by the residual U-net architecture of the convolutional neural network (CNN). We propose a residual dilated causal convolutional neural network (Res-DCCNN) with nonlinear attention for multi-step-ahead wind speed forecasting. Our model can outperform long-term short-term memory networks (LSTM), gated recurrent units (GRU), and Res-DCCNN using sliding window validation techniques for 50-step-ahead wind speed prediction. We tested the performance of the proposed model on six real-world wind speed datasets with different probability distributions to confirm its effectiveness, and using several error metrics, we demonstrated that our proposed model was robust, precise, and applicable to real-world cases.

show abstract

“…Specifically, they have used CNN as the feature extractor of short texts and SVM as the classifier, and SVMCNN shows a better performance than each of CNN and SVM. Liu et al have proposed an attention-gated CNN for the sentence classification by generating attention weights from the feature's context windows before the pooling layer [36], which shows a better performance than standard CNN models.…”

Section: Cnnmentioning

confidence: 99%

A New Text Classification Model Based on Contrastive Word Embedding for Detecting Cybersecurity Intelligence in Twitter

2020

View full text Add to dashboard Cite

Detecting cybersecurity intelligence (CSI) on social media such as Twitter is crucial because it allows security experts to respond cyber threats in advance. In this paper, we devise a new text classification model based on deep learning to classify CSI-positive and -negative tweets from a collection of tweets. For this, we propose a novel word embedding model, called contrastive word embedding, that enables to maximize the difference between base embedding models. First, we define CSI-positive and -negative corpora, which are used for constructing embedding models. Here, to supplement the imbalance of tweet data sets, we additionally employ the background knowledge for each tweet corpus: (1) CVE data set for CSI-positive corpus and (2) Wikitext data set for CSI-negative corpus. Second, we adopt the deep learning models such as CNN or LSTM to extract adequate feature vectors from the embedding models and integrate the feature vectors into one classifier. To validate the effectiveness of the proposed model, we compare our method with two baseline classification models: (1) a model based on a single embedding model constructed with CSI-positive corpus only and (2) another model with CSI-negative corpus only. As a result, we indicate that the proposed model shows high accuracy, i.e., 0.934 of F1-score and 0.935 of area under the curve (AUC), which improves the baseline models by 1.76∼6.74% of F1-score and by 1.64∼6.98% of AUC.

show abstract

An attention-gated convolutional neural network for sentence classification

Cited by 37 publications

References 36 publications

MGL-CNN: A Hierarchical Posts Representations Model for Identifying Depressed Individuals in Online Forums

MGL-CNN: A Hierarchical Posts Representations Model for Identifying Depressed Individuals in Online Forums

Multi-Step Short-Term Wind Speed Prediction Using a Residual Dilated Causal Convolutional Network with Nonlinear Attention

A New Text Classification Model Based on Contrastive Word Embedding for Detecting Cybersecurity Intelligence in Twitter

Contact Info

Product

Resources

About