Evaluating the quality and safety of health-related apps and e-tools: Adapting the Mobile App Rating Scale and developing a quality assurance protocol

Abstract

Background Whilst apps and e-tools have tremendous potential as low-cost, scalable mental health intervention and prevention tools, it is essential that consumers and health professionals have a means by which to evaluate their quality and safety. Objective This study aimed to: 1) adapt the original Mobile App Rating Scale (MARS) in order to be appropriate for the evaluation of both mobile phone applications as well as e-tools; 2) test the reliability of the revised scale; and 3) develop a quality assurance protocol for identifying and rating new apps and e-tools to determine appropriateness for use in clinical practice. Methods The MARS was adapted to include items specific to health-related apps and e-tools, such as the availability of resources, strategies for self-management, and quality information. The 41 apps and e-tools in the standard youth configuration of the InnoWell Platform, a digital tool designed to support or enhance mental health service delivery, were independently rated by two expert raters using the A-MARS. Cronbach's alpha was used to calculate the internal consistency and interclass correlation coefficients were used to calculate interrater reliability. Results The A-MARS was shown to be a reliable scale with acceptable to excellent internal consistency and moderate to excellent interrater reliability across the subscales. Given the ever-increasing number of health information technologies on the market, a protocol to identify and rate new apps and e-tools for potential clinical use is presented. Conclusions Whilst the A-MARS is a useful tool to guide health professionals as they explore available apps and e-tools for potential clinical use, the training, time, and skill required to use it effectively may be prohibitive. As such, health professionals and services are likely to benefit from including a digital navigator as part of the care team to assist in selecting and rating apps and e-tools, increasing the usability of the data, and technology troubleshooting. When selecting, evaluating and/or recommending apps and e-tools to consumers, it is important to consider: 1) the availability of explicit strategies to set, monitor and review SMART goals; 2) the accessibility of credible, user friendly information and resources from reputable sources; 3) evidence of effectiveness; and 4) interoperability with other health information technologies.

The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality

“…However, randomized controlled trials which have used app‐based interventions alongside human coaching to bolster their usage in community patients with depression have produced more robust evidence 42 , suggesting that human engagement in supporting app‐based interventions is critical. New roles such as digital navigators to support app use in mental health care may provide one solution to offer human support without overburdening the clinician 125‐128 (see below).…”

Stimuli Influencing Engagement, Satisfaction, and Intention to Use Telemedicine Services: An Integrative Model

“…In line with earlier studies, we used the S-O-R framework to investigate the internal mechanisms between external drivers and behavioral intentions for telemedicine services. Since telemedicine is a technology-integrated platform, its nature, amenities, and functionality are essential for assessing its quality and safety [ 32 ]. Thus, the platform infrastructure and characteristics function as the stimuli that kindle the users’ internal motivations.…”

“…More importantly, we dissect the mediating mechanisms, such as engagement and satisfaction, through which stimuli can achieve this. We establish the current research model by using stimulus-organism-response (S-O-R) theory [ 21 ], wherein we adjoin the Unified Theory of Acceptance and Use of Technology (UTAUT2) model [ 30 ], the expectation confirmation model (ECM) [ 31 ], and the Adapted-Mobile App Rating Scale (A-MARS) [ 32 ]. Many researchers have empirically validated their findings by applying these theoretical lenses but have not integrated these approaches in the telemedicine context.…”

Identifikation rheumatologischer Gesundheits-Apps im Apple App Store mit der Methode der „semiautomatischen retrospektiven App Store-Analyse“

“…Verschiedene Analysetools, wie z. B. die entwickelte Mobile App Rating Scale (MARS), sollen ebenfalls helfen, relevante, qualitativ hochwertige Apps zu identifizieren [ 2 , 20 , 27 ]. Für einen Teil der mit SARASA identifizierten Apps sind entsprechende MARS-Evaluationen publiziert [ 2 ].…”