Predicting Generalized Anxiety Disorder From Impromptu Speech Transcripts Using Context-Aware Transformer-Based Neural Networks: Model Evaluation Study

Abstract

Background The ability to automatically detect anxiety disorders from speech could be useful as a screening tool for an anxiety disorder. Prior studies have shown that individual words in textual transcripts of speech have an association with anxiety severity. Transformer-based neural networks are models that have been recently shown to have powerful predictive capabilities based on the context of more than one input word. Transformers detect linguistic patterns and can be separately trained to make specific predictions based on these patterns. Objective This study aimed to determine whether a transformer-based language model can be used to screen for generalized anxiety disorder from impromptu speech transcripts. Methods A total of 2000 participants provided an impromptu speech sample in response to a modified version of the Trier Social Stress Test (TSST). They also completed the Generalized Anxiety Disorder 7-item (GAD-7) scale. A transformer-based neural network model (pretrained on large textual corpora) was fine-tuned on the speech transcripts and the GAD-7 to predict whether a participant was above or below a screening threshold of the GAD-7. We reported the area under the receiver operating characteristic curve (AUROC) on the test data and compared the results with a baseline logistic regression model using the Linguistic Inquiry and Word Count (LIWC) features as input. Using the integrated gradient method to determine specific words that strongly affect the predictions, we inferred specific linguistic patterns that influence the predictions. Results The baseline LIWC-based logistic regression model had an AUROC value of 0.58. The fine-tuned transformer model achieved an AUROC value of 0.64. Specific words that were often implicated in the predictions were also dependent on the context. For example, the first-person singular pronoun “I” influenced toward an anxious prediction 88% of the time and a nonanxious prediction 12% of the time, depending on the context. Silent pauses in speech, also often implicated in predictions, influenced toward an anxious prediction 20% of the time and a nonanxious prediction 80% of the time. Conclusions There is evidence that a transformer-based neural network model has increased predictive power compared with the single word–based LIWC model. We also showed that the use of specific words in a specific context—a linguistic pattern—is part of the reason for the better prediction. This suggests that such transformer-based models could play a useful role in anxiety screening systems.

Automated speech analysis for risk detection of depression, anxiety, insomnia, and fatigue: Algorithm Development and Validation Study

“…Our recruitment and involvement of participants was in person. For medium-sized datasets, below 1000 participants, such as ours, it can be observed that web-based crowd-sourced recruitment of participants needs more training data to yield the same level of model accuracy [62, 64], as models trained on speech with face-to-face recruitment. This was also observed in a large study, with over 6000 participants, with online data recruitment for risk detection in the general population in American English [63].…”

Investigating Health and Well-Being Challenges Faced by an Aging Workforce in the Construction and Nursing Industries: Computational Linguistic Analysis of Twitter Data

“…Computational linguistic tools and algorithms are able to reliably predict risks of future mental illness [ 47 , 48 ]. The patterns observed in this study strongly suggested that nurses were experiencing stress and negative emotions, and this was more extreme in younger nurses.…”

Comprehensive review of Transformer‐based models in neuroscience, neurology, and psychiatry

“…Li et al [91] further improved ASD diagnosis by using a Transformer-based model on raw fMRI data, achieving a diagnostic accuracy of 74.18%. Teferra et al [20] showed that Transformers effectively enhanced anxiety disorder diagnosis. Li et al [18] introduced an accurate EEG-based method for detecting SCZ, employing a novel mapping approach and Levit classifier.…”

“…Specifically, in the field of neuroscience, Transformers have been used for tasks such as analyzing changes in human brain structure [6][7][8] and functional connectivity (FC), [9,10] identifying genetic variants underlying neurological phenotypes, [11,12] and decoding emotional patterns. [13] In the field of neurology and psychiatry, Transformers have been widely adopted for various neurological and psychiatric disorders, including Alzheimer's disease (AD), [14] Parkinson's disease (PD), [15] epilepsy, [16] stroke, [17] schizophrenia (SCZ), [18] depression, [19] and anxiety, [20] as well as for forecasting treatment outcomes [21] and predicting drug responses. [22] 1.…”