Assessing mental health apps marketplaces with objective metrics from 29,190 data points from 278 apps

Abstract

Objective: Utilizing a standard framework that may help clinicians and patients to identify relevant mental health apps, we sought to gain a comprehensive picture of the space by searching for, downloading, and reviewing 278 mental health apps from both the iOS and Android stores. Methods: 278 mental health apps from the Apple iOS store and Google Play store were downloaded and reviewed in a standardized manner by trained app raters using a validated framework. Apps were evaluated with this framework comprising 105 questions and covering app origin and accessibility, privacy and security, inputs and outputs, clinical foundation, features and engagement style, and interoperability. Results: Our results confirm that app stars and downloads-even for the most popular apps by these metrics-did not correlate with more clinically relevant metrics related to privacy/security, effectiveness, and engagement. Most mental health apps offer similar functionality, with 16.5% offering both mood tracking and journaling and 7% offering psychoeducation, deep breathing, mindfulness, journaling, and mood tracking. Only 36.4% of apps were updated with a 100-day window, and 7.5% of apps had not been updated in four years. Conclusion: Current app marketplace metrics commonly used to evaluate apps do not offer an accurate representation of individual apps or a comprehensive overview of the entire space. The majority of apps overlap in terms of features offered, with many domains and other features not well represented. Selecting an appropriate app continues to require personal matching given no clear trends or guidance offered by quantitative metrics alone. K E Y W O R D S digital health, apps, internet therapy Significant outcomes• App store star ratings and number of downloads are not associated with app features, function, or evidence. • Mood/symptom tracking, journaling, mindfulness, psychoeducation, and deep breathing are the five most common offering of mental health apps. • Nearly 1/3 of apps may update in a 100-day period, suggesting need to for constant evaluation.

Evidence of User-Expert Gaps in Health App Ratings and Implications for Practice

“…One important strength of this study is its sample size, which is significantly higher than similar studies (1,12,23), although it should be noted that this includes overlaps of the same health apps between stores. This, combined with the fact that both physical health and mental health are covered, makes this study one of the most comprehensive estimates of the relationship between user and expert ratings across the marketplace.…”

“…There is also partial evidence of a weak correlation between user ratings and clinical utility rated by experts ( 1 ), suggesting that the two reflect competing priorities [for similar results regarding apps for women's health, see ( 9 )]. Only Lagan et al explicitly compared five-star user ratings provided in commercial stores with expert ratings from a curated library of 278 apps for mental health ( 12 ). They confirmed user-expert gaps for key clinical metrics such as privacy, security, efficacy, and engagement.…”

Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management

“…The MIND framework consists of 105 objective questions that map onto the APA framework and principles of medical ethics, and is divided into six sections: app origin and functionality (background information and accessibility), inputs and outputs (what types of data are collected and generated), privacy and security (data use and sharing, patient safety), clinical foundation (the existence of peer reviewed literature describing the feasibility and efficacy of reviewed apps), features and engagement style (app interface and overall functionality), interoperability and data sharing (ability to export/share data for personal or clinical use). The MIND framework has been used to describe the quality and safety of mental health apps for schizophrenia (Lagan et al 2021a ), bipolar disorder (Lagan et al 2020a ), Spanish language speakers (Muñoz et al 2021 ), peripartum affective disorders (Feldman et al 2021 ), and top-returned apps for mental health related searches (Lagan et al 2021b ).…”

“…It should be noted that as the MIND framework is intended for use by non-specialists (including people with lived experience, healthcare administrators, and policymakers) to provide a general assessment of the quality of a given app, a formal systematic review is not required to answer items in the clinical foundation section, nor are raters instructed in evaluating the quality of scientific literature (Lagan et al 2021b ). In a similar vein, the MIND framework generates a descriptive summary of app characteristics; no total score or quantitative summary is provided.…”

Psychiatry in the Digital Age: A Blessing or a Curse?

“…Furthermore, Larsen et al showed that there is a lack of evidence from app-specific studies, and many mental health apps publicly available in app stores describe techniques for which there is no clear evidence in the literature [79]. This overall lack of evidence was confirmed by Lagan et al in a recent review of 278 publicly available mental health apps [80].…”