Self-Assessment of Having COVID-19 With the Corona Check mHealth App

Beierle, Felix; Allgaier, Johannes; Stupp, Carolin; Keil, Thomas; Schlee, Winfried; Schobel, Johannes; Vogel, Carsten; Haug, Fabian; Haug, Julian; Holfelder, Marc; Langguth, Berthold; Langguth, Jana; Riens, Burgi; King, Ryan S.; Mulansky, Lena; Schickler, Marc; Stach, Michael; Heuschmann, Peter U.; Wildner, Manfred; Greger, Helmut; Reichert, Manfred; Kestler, Hans A.; Pryss, Rüdiger

doi:10.1109/jbhi.2023.3264999

Cited by 4 publications

(1 citation statement)

References 45 publications

(40 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There has been some prior research on the public perception or public sentiments about the topic of mental health during the pandemic. It is typically based on smaller-scale data collected via surveys [2][3][4][5][6][7][8][9][10][11] or apps [12][13][14]. Analyzing the sentiments of a larger segment of the population regarding mental health can help public health officials gauge the direction of overall trends.…”

Section: Introductionmentioning

confidence: 99%

Sentiments about Mental Health on Twitter—Before and during the COVID-19 Pandemic

Beierle,

Pryss,

Aizawa

2023

Healthcare

Self Cite

View full text Add to dashboard Cite

During the COVID-19 pandemic, the novel coronavirus had an impact not only on public health but also on the mental health of the population. Public sentiment on mental health and depression is often captured only in small, survey-based studies, while work based on Twitter data often only looks at the period during the pandemic and does not make comparisons with the pre-pandemic situation. We collected tweets that included the hashtags #MentalHealth and #Depression from before and during the pandemic (8.5 months each). We used LDA (Latent Dirichlet Allocation) for topic modeling and LIWC, VADER, and NRC for sentiment analysis. We used three machine-learning classifiers to seek evidence regarding an automatically detectable change in tweets before vs. during the pandemic: (1) based on TF-IDF values, (2) based on the values from the sentiment libraries, (3) based on tweet content (deep-learning BERT classifier). Topic modeling revealed that Twitter users who explicitly used the hashtags #Depression and especially #MentalHealth did so to raise awareness. We observed an overall positive sentiment, and in tough times such as during the COVID-19 pandemic, tweets with #MentalHealth were often associated with gratitude. Among the three classification approaches, the BERT classifier showed the best performance, with an accuracy of 81% for #MentalHealth and 79% for #Depression. Although the data may have come from users familiar with mental health, these findings can help gauge public sentiment on the topic. The combination of (1) sentiment analysis, (2) topic modeling, and (3) tweet classification with machine learning proved useful in gaining comprehensive insight into public sentiment and could be applied to other data sources and topics.

show abstract

Section: Introductionmentioning

confidence: 99%

Sentiments about Mental Health on Twitter—Before and during the COVID-19 Pandemic

Beierle,

Pryss,

Aizawa

2023

Healthcare

Self Cite

View full text Add to dashboard Cite

show abstract

Practical approaches in evaluating validation and biases of machine learning applied to mobile health studies

Allgaier,

Pryss

2024

Commun Med

Self Cite

View full text Add to dashboard Cite

Background Machine learning (ML) models are evaluated in a test set to estimate model performance after deployment. The design of the test set is therefore of importance because if the data distribution after deployment differs too much, the model performance decreases. At the same time, the data often contains undetected groups. For example, multiple assessments from one user may constitute a group, which is usually the case in mHealth scenarios. Methods In this work, we evaluate a model’s performance using several cross-validation train-test-split approaches, in some cases deliberately ignoring the groups. By sorting the groups (in our case: Users) by time, we additionally simulate a concept drift scenario for better external validity. For this evaluation, we use 7 longitudinal mHealth datasets, all containing Ecological Momentary Assessments (EMA). Further, we compared the model performance with baseline heuristics, questioning the essential utility of a complex ML model. Results Hidden groups in the dataset leads to overestimation of ML performance after deployment. For prediction, a user’s last completed questionnaire is a reasonable heuristic for the next response, and potentially outperforms a complex ML model. Because we included 7 studies, low variance appears to be a more fundamental phenomenon of mHealth datasets. Conclusions The way mHealth-based data are generated by EMA leads to questions of user and assessment level and appropriate validation of ML models. Our analysis shows that further research needs to follow to obtain robust ML models. In addition, simple heuristics can be considered as an alternative for ML. Domain experts should be consulted to find potentially hidden groups in the data.

show abstract

Automating the Development of Stress Detection Systems

Beierle,

Pryss

2023

2023 Congress in Computer Science, Computer Engineering, &Amp;amp; Applied Computing (CSCE)

View full text Add to dashboard Cite

Self-Assessment of Having COVID-19 With the Corona Check mHealth App

Cited by 4 publications

References 45 publications

Sentiments about Mental Health on Twitter—Before and during the COVID-19 Pandemic

Sentiments about Mental Health on Twitter—Before and during the COVID-19 Pandemic

Practical approaches in evaluating validation and biases of machine learning applied to mobile health studies

Automating the Development of Stress Detection Systems

Contact Info

Product

Resources

About