Smartphone-Detected Ambient Speech and Self-Reported Measures of Anxiety and Depression: Exploratory Observational Study

Abstract

Background The ability to objectively measure the severity of depression and anxiety disorders in a passive manner could have a profound impact on the way in which these disorders are diagnosed, assessed, and treated. Existing studies have demonstrated links between both depression and anxiety and the linguistic properties of words that people use to communicate. Smartphones offer the ability to passively and continuously detect spoken words to monitor and analyze the linguistic properties of speech produced by the speaker and other sources of ambient speech in their environment. The linguistic properties of automatically detected and recognized speech may be used to build objective severity measures of depression and anxiety. Objective The aim of this study was to determine if the linguistic properties of words passively detected from environmental audio recorded using a participant’s smartphone can be used to find correlates of symptom severity of social anxiety disorder, generalized anxiety disorder, depression, and general impairment. Methods An Android app was designed to collect periodic audiorecordings of participants’ environments and to detect English words using automatic speech recognition. Participants were recruited into a 2-week observational study. The app was installed on the participants’ personal smartphones to record and analyze audio. The participants also completed self-report severity measures of social anxiety disorder, generalized anxiety disorder, depression, and functional impairment. Words detected from audiorecordings were categorized, and correlations were measured between words counts in each category and the 4 self-report measures to determine if any categories could serve as correlates of social anxiety disorder, generalized anxiety disorder, depression, or general impairment. Results The participants were 112 adults who resided in Canada from a nonclinical population; 86 participants yielded sufficient data for analysis. Correlations between word counts in 67 word categories and each of the 4 self-report measures revealed a strong relationship between the usage rates of death-related words and depressive symptoms (r=0.41, P<.001). There were also interesting correlations between rates of word usage in the categories of reward-related words with depression (r=–0.22, P=.04) and generalized anxiety (r=–0.29, P=.007), and vision-related words with social anxiety (r=0.31, P=.003). Conclusions In this study, words automatically recognized from environmental audio were shown to contain a number of potential associations with severity of depression and anxiety. This work suggests that sparsely sampled audio could provide relevant insight into individuals’ mental health.

Automated Screening for Social Anxiety, Generalized Anxiety, and Depression From Objective Smartphone-Collected Data: Cross-sectional Study

“…We use LIWC to count the percentage of all words collected from the person that fall within LIWC's death category and refer to this as the death-related words feature. In our previous work, we showed that the incidence of death-related words has been shown to have an association with symptoms of SAD, GAD, and depression [35]. The previous work of Di Matteo et al [35] also contains further methodological and implementation details of this LIWC-based feature extraction.…”

“…In our previous work, we showed that the incidence of death-related words has been shown to have an association with symptoms of SAD, GAD, and depression [35]. The previous work of Di Matteo et al [35] also contains further methodological and implementation details of this LIWC-based feature extraction.…”

Acoustic and Linguistic Features of Impromptu Speech and Their Association With Anxiety: Validation Study

“…Di Matteo et al [ 23 ] explored the relationship between linguistic features of speech and anxiety. Their work used passively collected intermittent samples of audio data from participants’ smartphones, collected over a 2-week period, as input.…”

Exploring the digital footprint of depression: a PRISMA systematic literature review of the empirical evidence

“…Studies used a range of technologies to collect active/subjective, passive/objective, and mixed (i.e., active/subjective and passive/objective) data. Studies employing passively/objectively collected data often produced predictive models with high accuracy in the detection of depression severity involving significant predictors such as geospatial movement, sleep duration, delayed sleep phase, circadian rhythm, audio features, language, accelerometer oscillation, and light exposure during bedtime [ 12 , 29 , 49 , 50 , 52 , 56 ]. Considering the type of technology, reviewed studies employed mobile technology (handheld IT devices such as smartphones, palmtops, tablets, laptops, etc.…”

“…Studies assessing mood include [ 1 , 2 , 6 , 8 , 12 , 15 , 18 , 21 , 24 , 27 , 29 , 30 , 31 , 33 , 35 , 37 , 40 , 45 , 46 , 50 , 51 , 66 , 67 , 71 , 72 . 73 , 77 , 79 , 80 , 82 , 86 , 87 , 88 , 89 , 94 , 100 , 102 , 103 , 104 , 105 , 109 , 114 , 121 , 122 , 125 , 130 , 131 , 135 , 136 , 138 , 139 , 141 , 142 , 144 , 147 , 148 , 150 , 151 , 172 , 173 , 174 , 181 , 183 , 186 , 188 , 190 , 191 , 192 , 193 , 194 …”