Evaluating User Feedback for an Artificial Intelligence–Enabled, Cognitive Behavioral Therapy–Based Mental Health App (Wysa): Qualitative Thematic Analysis

Abstract

Background Digital mental health apps are rapidly becoming a common source of accessible support across the world, but their effectiveness is often influenced by limited helpfulness and engagement. Objective This study’s primary objective was to analyze feedback content to understand users’ experiences with engaging with a digital mental health app. As a secondary objective, an exploratory analysis captured the types of mental health app users. Methods This study utilized a user-led approach to understanding factors for engagement and helpfulness in digital mental health by analyzing feedback (n=7929) reported on Google Play Store about Wysa, a mental health app (1-year period). The analysis of keywords in the user feedback categorized and evaluated the reported user experience into the core domains of acceptability, usability, usefulness, and integration. The study also captured key deficits and strengths of the app and explored salient characteristics of the types of users who benefit from accessible digital mental health support. Results The analysis of user feedback found the app to be overwhelmingly positively reviewed (6700/7929, 84.50% 5-star rating). The themes of engaging exercises, interactive interface, and artificial intelligence (AI) conversational ability indicated the acceptability of the app, while the nonjudgmentality and ease of conversation highlighted its usability. The app’s usefulness was portrayed by themes such as improvement in mental health, convenient access, and cognitive restructuring exercises. Themes of privacy and confidentiality underscored users’ preference for the integrated aspects of the app. Further analysis revealed 4 predominant types of individuals who shared app feedback on the store. Conclusions Users reported therapeutic elements of a comfortable, safe, and supportive environment through using the digital mental health app. Digital mental health apps may expand mental health access to those unable to access traditional forms of mental health support and treatments.

Evaluating a Digital Mental Health Intervention (Wysa) for Workers’ Compensation Claimants

“…Furthermore, participants with high severity injuries, which are associated with greater psychological morbidity, 35 were also statistically more likely to qualify for the study and use the app for longer than those with less severe injuries. The retention potentially indicates that the app was able to facilitate a supportive space for those with heightened needs, as also indicated by an acceptability study of user reviews for Wysa 26 …”

“…The retention potentially indicates that the app was able to facilitate a supportive space for those with heightened needs, as also indicated by an acceptability study of user reviews for Wysa. 26 …”

“…The retention potentially indicates that the app was able to facilitate a supportive space for those with heightened needs, as also indicated by an acceptability study of user reviews for Wysa. 26 For users who completed usage within the study period, the mean engagement in the study was 10.8 sessions and the mean retention period was 35 days, and over 153.9 days of app usage for those who continued utilization. Evidence suggests an average retention period between 4 and 16 days in most digital health applications.…”

“…22,23 CAs have been found to be effective in promoting stress management techniques, 24 and are feasible as an intervention for pain self-management. 25 An analysis of user feedback on Wysa, an AI-based CA, has indicated high levels of acceptability, with users highlighting its nonjudgmental approach and easy conversational style, 26 which enables the creation of a therapeutic alliance. 27 An evaluation of a novel, customized version of Wysa built to facilitate pain acceptance and behavioral activation for orthopedic patients with chronic musculoskeletal pain found high retention and engagement rates among users, with high engagement users reporting greater improvements in anxiety scores than low engagement users.…”

Understanding Digital Mental Health Needs and Usage With an Artificial Intelligence–Led Mental Health App (Wysa) During the COVID-19 Pandemic: Retrospective Analysis

“…Additionally, Wysa has been found to improve mental health outcomes in chronic conditions [ 20 , 21 ], with high engagement and adherence [ 22 ]. It was found that users were able to establish a strong therapeutic alliance with the AI-based conversational agent that was comparable to a human therapeutic bond [ 23 ], further confirming the positive feedback from users [ 24 ]. During the pandemic, Wysa also released 2 tool packs (or intervention packages) named “Health Anxiety” and “Remote Wellness,” which were contextualized to the pandemic.…”