A Novel Text Mining Approach for Mental Health Prediction Using Bi-LSTM and BERT Model

Abstract

With the current advancement in the Internet, there has been a growing demand for building intelligent and smart systems that can efficiently address the detection of health-related problems on social media, such as the detection of depression and anxiety. These types of systems, which are mainly dependent on machine learning techniques, must be able to deal with obtaining the semantic and syntactic meaning of texts posted by users on social media. The data generated by users on social media contains unstructured and unpredictable content. Several systems based on machine learning and social media platforms have recently been introduced to identify health-related problems. However, the text representation and deep learning techniques employed provide only limited information and knowledge about the different texts posted by users. This is owing to a lack of long-term dependencies between each word in the entire text and a lack of proper exploitation of recent deep learning schemes. In this paper, we propose a novel framework to efficiently and effectively identify depression and anxiety-related posts while maintaining the contextual and semantic meaning of the words used in the whole corpus when applying bidirectional encoder representations from transformers (BERT). In addition, we propose a knowledge distillation technique, which is a recent technique for transferring knowledge from a large pretrained model (BERT) to a smaller model to boost performance and accuracy. We also devised our own data collection framework from Reddit and Twitter, which are the most common social media sites. Finally, we employed word2vec and BERT with Bi-LSTM to effectively analyze and detect depression and anxiety signs from social media posts. Our system surpasses other state-of-the-art methods and achieves an accuracy of 98% using the knowledge distillation technique.

Network Public Opinion Risk Prediction and Judgment Based on Deep Learning: A Model of Text Sentiment Analysis

“…To sum up, in order to better solve the problem of online public opinion sentiment classifcation, this paper proposes a BCBL model that combines BERT, CNN, and Bi-LSTM technologies [20][21][22][23][24].…”

DDdeep: deep learning-based text analysis for depression illness detection on social media posts

“…As the results of Table 5 show, the proposed method has provided a successful performance in identifying the depressed users over the methods under comparison. For example, the proposed method provides about 2% improvement in precision index, 1% in recall and 3% in F1 score in comparision with the method of the BERT + knowledge distillation (Zeberga et al, 2022). It is also seen in Table 5 that the highest precision (0.78), recall (0.70) and F1 score (0.73) are provided by the proposed method, and this improvement can be a reason to prove the superiority of the proposed method over to other compared methods.…”

“…al., 2021;Rhanoui et al, 2019;Zeberga et al, 2022). The rst convolutional layer extracts the features of each of the user's posts by applying a lter in the sliding window, and then by applying the merging layer, only the important features are kept.…”

Next Generation Infectious Diseases Monitoring Gages via Incremental Federated Learning: Current Trends and Future Possibilities

“…A "federated learning" technique involves training an algorithm without exchanging information between servers containing local data samples or other clustered edge gadgets as compared to conventional centralized machine learning methods, in which all local datasets are transferred to a single server and trained using the master model that will further globally train the peer nodes [34]. Data access rights, data privacy, heterogeneous data access, and security are factors that can be addressed with the help of federated learning.…”