Assessing the quality of mobile applications in chronic disease management: a scoping review

Agarwal, Payal; Gordon, Dara; Griffith, Janessa; Kithulegoda, Natasha; Witteman, Holly O.; Bhatia, R Sacha; Kushniruk, André; Borycki, Elizabeth M.; Lamothe, Lise; Springall, Elena; Shaw, James

doi:10.1038/s41746-021-00410-x

Cited by 53 publications

(51 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nonetheless, none of these was implemented in the peer-reviewed studies identified, emphasising the preference for parallel-group RCTs or checklists of quality criteria that show significant variability and no clear method for their development 132 .…”

Section: Discussionmentioning

confidence: 99%

Development features and study characteristics of mobile health apps in the management of chronic conditions: a systematic review of randomised trials

et al. 2021

View full text Add to dashboard Cite

COVID-19 pandemic challenges have accelerated the reliance on digital health fuelling the expanded incorporation of mobile apps into healthcare services, particularly for the management of long-term conditions such as chronic diseases (CDs). However, the impact of health apps on outcomes for CD remains unclear, potentially owing to both the poor adoption of formal development standards in the design process and the methodological quality of studies. A systematic search of randomised trials was performed on Medline, ScienceDirect, the Cochrane Library and Scopus to provide a comprehensive outlook and review the impact of health apps on CD. We identified 69 studies on diabetes (n = 29), cardiovascular diseases (n = 13), chronic respiratory diseases (n = 13), cancer (n = 10) or their combinations (n = 4). The apps rarely adopted developmental factors in the design stage, with only around one-third of studies reporting user or healthcare professional engagement. Apps differed significantly in content, with a median of eight behaviour change techniques adopted, most frequently pertaining to the ‘Feedback and monitoring’ (91%) and ‘Shaping knowledge’ (72%) categories. As for the study methodologies, all studies adopted a traditional randomised control trial (RCT) design, with relatively short follow-ups and limited sample sizes. Findings were not significant for the majority of studies across all CD, with most RCTs revealing a high risk of bias. To support the adoption of apps for CD management, this review reinforces the need for more robust development and appropriate study characteristics to sustain evidence generation and elucidate whether study results reflect the true benefits of apps or a biased estimate due to unsuitable designs.

show abstract

Section: Discussionmentioning

confidence: 99%

Development features and study characteristics of mobile health apps in the management of chronic conditions: a systematic review of randomised trials

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Regarding methods for evaluating mHealth tools, a recent scoping review summarizing quality assessment methods for mobile apps in chronic disease management found minimal agreement on the most appropriate criteria for evaluating mobile apps, with only 18% (12/65) of apps including evidence-based health information, 22% (14/65) using a behavioral change framework, and 37% (24/65) applying usability metrics as quality criteria for app assessment [ 51 ]. On the basis of the gaps identified in the analysis and practice methods for evaluating mobile apps, the authors proposed 3 primary goals for building quality criteria for app assessment: (1) prioritize existing evidence and knowledge against ease of assessment (eg, using patient-reported outcomes vs app ratings), (2) emphasize principles of behavior change theory, and (3) explicitly incorporate the patient perspective.…”

Section: Discussionmentioning

confidence: 99%

Patients’ Use of Mobile Health for Self-management of Knee Osteoarthritis: Results of a 6-Week Pilot Study

Shewchuk¹,

Green²,

Barber³

et al. 2021

JMIR Form Res

View full text Add to dashboard Cite

Background In a previous study, a prototype mobile health (mHealth) app was co-designed with patients, family physicians, and researchers to enhance self-management and optimize conservative management for patients with mild to moderate knee osteoarthritis (OA). Objective This study aims to evaluate the overall usability, quality, and effectiveness of the mHealth app prototype for aiding knee OA self-management from the perspectives of patients with OA and health care providers (HCPs). Methods Using methods triangulation of qualitative and quantitative data, we conducted a pilot evaluation of an mHealth app prototype that was codeveloped with patients and HCPs. We recruited adult patients aged ≥20 years with early knee OA (n=18) who experienced knee pain on most days of the month at any time in the past and HCPs (n=7) to participate. In the qualitative assessment, patient and HCP perspectives were elicited on the likeability and usefulness of app features and functionalities and the perceived impact of the app on patient-HCP communication. The quantitative assessment involved evaluating the app using usability, quality, and effectiveness metrics. Patient baseline assessments included a semistructured interview and survey to gather demographics and assess the quality of life (European Quality-of-Life 5-Dimension 5-Level Questionnaire [EQ-5D-5L]) and patient activation (patient activation measure [PAM]). Following the 6-week usability trial period, a follow-up survey assessed patients’ perceptions of app usability and quality and longitudinal changes in quality of life and patient activation. Semistructured interviews and surveys were also conducted with HCPs (n=7) at baseline to evaluate the usability and quality of the app prototype. Results Interviews with patients and HCPs revealed overall positive impressions of the app prototype features and functionalities related to likeability and usefulness. Between the baseline and follow-up patient assessments, the mean EQ-5D-5L scores improved from 0.77 to 0.67 (P=.04), and PAM scores increased from 80.4 to 87.9 (P=.01). Following the 6-week evaluation, patients reported a mean System Usability Scale (SUS) score of 57.8, indicating marginal acceptability according to SUS cutoffs. The mean number of goals set during the usability period was 2.47 (SD 3.08), and the mean number of activities completed for knee OA self-management during the study period was 22.2 (SD 17.8). Spearman rank correlation (rs) calculations revealed that the follow-up PAM scores were weakly correlated (rs=−0.32) with the number of goals achieved and the number (rs=0.19) of activities performed during the 6-week usability period. HCPs reported a mean SUS score of 39.1, indicating unacceptable usability. Conclusions This evidence-based and patient-centered app prototype represents a potential use of mHealth for improving outcomes and enhancing conservative care by promoting patient activation and patient-HCP communication regarding OA management. However, future iterations of the app prototype are required to address the limitations related to usability and quality.

show abstract

“…However, the constant creation and turnover of new apps can make this task difficult and time-consuming. Although syntheses of eHealth resources are becoming more common, their methodological and reporting quality require improvement so as to be more accessible to nonexperts [ 16 ]. Further, there continues to be significant variation in quality criteria employed for assessment, with no clear method for developing the included criteria.…”

Section: Introductionmentioning

confidence: 99%

Selecting and Evaluating Mobile Health Apps for the Healthy Life Trajectories Initiative: Development of the eHealth Resource Checklist

Vanderloo

Carsley

Agarwal

et al. 2021

JMIR Mhealth Uhealth

Self Cite

View full text Add to dashboard Cite

Background The ubiquity of smartphones and mobile devices in the general population presents an unprecedented opportunity for preventative health. Not surprisingly, the use of electronic health (eHealth) resources accessed through mobile devices in clinical trials is becoming more prevalent; the selection, screening, and collation of quality eHealth resources is necessary to clinical trials using these technologies. However, the constant creation and turnover of new eHealth resources can make this task difficult. Although syntheses of eHealth resources are becoming more common, their methodological and reporting quality require improvement so as to be more accessible to nonexperts. Further, there continues to be significant variation in quality criteria employed for assessment, with no clear method for developing the included criteria. There is currently no single existing framework that addresses all six dimensions of mobile health app quality identified in Agarwal et al’s recent scoping review (ie, basic descriptions of the design and usage of the resource; technical features and accessibility; health information quality; usability; evidence of impact; and user engagement and behavior change). In instances where highly systematic tactics are not possible (due to time constraints, cost, or lack of expertise), there may be value in adopting practical and pragmatic approaches to helping researchers and clinicians identify and disseminate e-resources. Objective The study aimed to create a set of guidelines (ie, a checklist) to aid the members of the Healthy Life Trajectories Initiative (HeLTI) Canada trial—a preconception randomized controlled clinical trial to prevent child obesity—to assist their efforts in searching, identifying, screening, and including selected eHealth resources for participant use in the study intervention. Methods A framework for searching, screening, and selecting eHealth resources was adapted from the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) checklist for systematic and scoping reviews to optimize the rigor, clarity, and transparency of the process. Details regarding searching, selecting, extracting, and assessing quality of eHealth resources are described. Results This study resulted in the systematic development of a checklist consisting of 12 guiding principles, organized in a chronological versus priority sequence to aid researchers in searching, screening, and assessing the quality of various eHealth resources. Conclusions The eHealth Resource Checklist will assist researchers in navigating the eHealth resource space by providing a mechanism to detail their process of developing inclusion criteria, identifying search location, selecting and reviewing evidence, extracting information, evaluating the quality of the evidence, and synthesizing the extracted evidence. The overarching goal of this checklist is to provide researchers or generalists new to the eHealth field with a tool that balances pragmatism with rigor and that helps standardize the process of searching and critiquing digital material—a particularly important aspect given the recent explosion of and reliance on eHealth resources. Moreover, this checklist may be useful to other researchers and practitioners developing similar health interventions.

show abstract

Assessing the quality of mobile applications in chronic disease management: a scoping review

Cited by 53 publications

References 53 publications

Development features and study characteristics of mobile health apps in the management of chronic conditions: a systematic review of randomised trials

Development features and study characteristics of mobile health apps in the management of chronic conditions: a systematic review of randomised trials

Patients’ Use of Mobile Health for Self-management of Knee Osteoarthritis: Results of a 6-Week Pilot Study

Selecting and Evaluating Mobile Health Apps for the Healthy Life Trajectories Initiative: Development of the eHealth Resource Checklist

Contact Info

Product

Resources

About