Using Acoustic Speech Patterns From Smartphones to Investigate Mood Disorders: Scoping Review

Abstract

Background Mood disorders are commonly underrecognized and undertreated, as diagnosis is reliant on self-reporting and clinical assessments that are often not timely. Speech characteristics of those with mood disorders differs from healthy individuals. With the wide use of smartphones, and the emergence of machine learning approaches, smartphones can be used to monitor speech patterns to help the diagnosis and monitoring of mood disorders. Objective The aim of this review is to synthesize research on using speech patterns from smartphones to diagnose and monitor mood disorders. Methods Literature searches of major databases, Medline, PsycInfo, EMBASE, and CINAHL, initially identified 832 relevant articles using the search terms “mood disorders”, “smartphone”, “voice analysis”, and their variants. Only 13 studies met inclusion criteria: use of a smartphone for capturing voice data, focus on diagnosing or monitoring a mood disorder(s), clinical populations recruited prospectively, and in the English language only. Articles were assessed by 2 reviewers, and data extracted included data type, classifiers used, methods of capture, and study results. Studies were analyzed using a narrative synthesis approach. Results Studies showed that voice data alone had reasonable accuracy in predicting mood states and mood fluctuations based on objectively monitored speech patterns. While a fusion of different sensor modalities revealed the highest accuracy (97.4%), nearly 80% of included studies were pilot trials or feasibility studies without control groups and had small sample sizes ranging from 1 to 73 participants. Studies were also carried out over short or varying timeframes and had significant heterogeneity of methods in terms of the types of audio data captured, environmental contexts, classifiers, and measures to control for privacy and ambient noise. Conclusions Approaches that allow smartphone-based monitoring of speech patterns in mood disorders are rapidly growing. The current body of evidence supports the value of speech patterns to monitor, classify, and predict mood states in real time. However, many challenges remain around the robustness, cost-effectiveness, and acceptability of such an approach and further work is required to build on current research and reduce heterogeneity of methodologies as well as clinical evaluation of the benefits and risks of such approaches.

Automatic Depression Detection Using Smartphone-Based Text-Dependent Speech Signals: Deep Convolutional Neural Network Approach

“…This study provided only the Concordance Correlation Coefficient (CCC) and Root Mean Square Error as an evaluation of the model for predicting the PHQ-9 score. The audio data set from the audio or visual emotion challenge and workshop 2019 detecting depression with artificial intelligence subchallenge, the Extended Distress Analysis Interview Corpus [ 62 ], was used as the text-independent setting. The Extended Distress Analysis Interview Corpus contains a multimodal data set of semistructured clinical interviews to evaluate the diagnosis of psychiatric stressors, such as depression, anxiety, and posttraumatic stress disorder.…”

“…However, our model could not be generalized for the prediction of PHQ-9 score and in spontaneous situations. Text-dependent read speech is more private and can be easily obtained with a smartphone compared with voluntary speech containing personal information [ 30 , 62 ]. Implementing these text-dependent speech tasks has the advantage of reducing acoustic variability and enabling more precise analysis because they acquire speech in a controlled manner and can standardize speech acquisition to produce consistent results in detecting depression.…”

Machine Learning Algorithms for Predicting Factitious Disorder Using the Learning Vector Quantization Method

“…Other studies that use machine learning (ML) to investigate mental health or substance use issues among lesbian, gay, bisexual, transgender, queer or questioning, and two-spirit (LGBTQ2S+) populations [5]. Occupational Depression Inventory (ODI), a measure designed to measure the severity of work-associated depressive symptoms and establish a provisional diagnosis of work-induced depression [6].…”

Zero-shot personalization of speech foundation models for depressed mood monitoring

“… 9 , 10 Typically, these take the form of mobile monitoring applications that utilize various sensors embedded in modern smartphones or wearables and have been shown to correlate with mental states. 10 , 11 , 12 Speech, as one of the biomarkers affected by different pathologies, 13 , 14 , 15 such as mood disorders 16 and depression, 17 , 18 , 19 can be used as a means to passively monitor patients. This can be done through the pervasive recording of daily life, 20 using minimalistic models deployed in edge devices, 21 monitoring telephone conversations, 22 or eliciting responses through human-computer interaction interfaces (e.g., computer games 23 ) in a naturalistic setting.…”