Hierarchical neural model with attention mechanisms for the classification of social media text related to mental health

Abstract

Mental health problems represent a major public health challenge. Automated analysis of text related to mental health is aimed to help medical decision-making, public health policies and to improve health care. Such analysis may involve text classification. Traditionally, automated classification has been performed mainly using machine learning methods involving costly feature engineering. Recently, the performance of those methods has been dramatically improved by neural methods. However, mainly Convolutional neural networks (CNNs) have been explored. In this paper, we apply a hierarchical Recurrent neural network (RNN) architecture with an attention mechanism on social media data related to mental health. We show that this architecture improves overall classification results as compared to previously reported results on the same data. Benefitting from the attention mechanism, it can also efficiently select text elements crucial for classification decisions, which can also be used for in-depth analysis.

Recent Advances in Using Natural Language Processing to Address Public Health Research Questions Using Social Media and ConsumerGenerated Data

“…Of the thirty-one mental health-related papers reviewed (see Table 8), thirteen involved the use of Reddit data [15][16][17][18][19][20][21][22][23][24][25][26][27], ten used Twitter data [18,24,[58][59][60][61][62][63][64][65], one used Instagram [18], three used Facebook [8,18,67], six used OHC data [70][71][72][73][74][75], and one used data derived from Weibo [76], with twenty-two of the papers utilising supervised machine learning methods [8, 16, 18, 20-22, 24, 25, 58-62, 65, 67, 70-76], and twelve papers utilising unsupervised machine learning [8, 15, 18-22, 27, 59, 60, 70, 72]. The majority of the papers reported on the use of classical machine learning approaches [8, 15, 16, 18-20, 22, 24, 25, 27, 58-62, 65, 67, 71, 73-76], with a minority using modern machine learning methods [18,21,22,67,70,72]. Four of the mental health papers reviewed utilised primarily lexicon...…”

Latent Topic Attention for Domain Classification

“…Recently, various attention mechanisms have been previously studied in text classification problems. Here, we consider two popular alternatives, soft attention [5][6][7] and hard attention [15].…”

“…Soft attention was widely used in text classification, such as intent detection [5], relation classification [6] and document classification [7]. Although there exists a little difference in the computations about the functions of score, the attention mechanism is to calculate the "soft" aligned attention weight from the output of the encoder, and then the output of the encoder is scaled according to the weight of attention:…”

“…Especially, the recurrent neural network (RNN) model has been successfully applied in many sequence learning problems [4], as it can be easy to deal with variable-length input utterances.  Recent approaches introduce attention mechanisms to focus the models on informative words [5][6][7]. However, to our knowledge, the parameter vectors of the attention in all of the existing attention approaches are trained with the domain tags.…”

Predicting Social Network Users with Depression from Simulated Temporal Data

“…In healthcare settings, huge amounts of textual data e.g., electronic health records increasingly generated are useful for researchers to develop text classifiers to identify information related to disease. Text classification models have been developed to label created content with categories, with some labelling social media posts related to users mental health [4,5].…”