Mobile health app usability and quality rating scales: a systematic review

Azad-Khaneghah, Peyman; Neubauer, Noelannah; Miguel-Cruz, Antonio; Liu, Huan

doi:10.1080/17483107.2019.1701103

Cited by 97 publications

(69 citation statements)

References 98 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Among scales/questionnaires, which constitute the technique most often reported, the most common usability assessment scales were the System Usability Scale [ 29 , 32 , 41 - 43 , 46 , 47 ] and the Post-Study System Usability Questionnaire [ 41 , 42 , 46 , 47 ]. The other scales/questionnaires include the Questionnaire for User Interaction Satisfaction [ 29 , 42 , 47 ], the Software Usability Measurement Inventory [ 32 , 42 ], the Usefulness, Satisfaction, and Ease of use Questionnaire [ 32 , 41 ], the Computer System Usability Questionnaire [ 32 , 47 ], the After-Scenario Questionnaire [ 46 , 47 ], the Perceived Useful and Ease of Use [ 32 ], the IsoMetrics usability inventory [ 32 ], the Health Information Technology Usability Evaluation Scale [ 32 ], the user Mobile Application Rating Scale [ 32 ]; the IBM ease of use [ 42 ], and the ISO 9241–11 Questionnaire [ 43 ]. In addition, several reviews have reported the use of nonvalidated questionnaires [ 32 , 41 , 43 , 46 ].…”

Section: Resultsmentioning

confidence: 99%

“…The other scales/questionnaires include the Questionnaire for User Interaction Satisfaction [ 29 , 42 , 47 ], the Software Usability Measurement Inventory [ 32 , 42 ], the Usefulness, Satisfaction, and Ease of use Questionnaire [ 32 , 41 ], the Computer System Usability Questionnaire [ 32 , 47 ], the After-Scenario Questionnaire [ 46 , 47 ], the Perceived Useful and Ease of Use [ 32 ], the IsoMetrics usability inventory [ 32 ], the Health Information Technology Usability Evaluation Scale [ 32 ], the user Mobile Application Rating Scale [ 32 ]; the IBM ease of use [ 42 ], and the ISO 9241–11 Questionnaire [ 43 ]. In addition, several reviews have reported the use of nonvalidated questionnaires [ 32 , 41 , 43 , 46 ]. One review reported that 26% of the included studies used a remote assessment of usability, where participants are in an uncontrolled environment [ 31 ].…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Procedures of User-Centered Usability Assessment for Digital Solutions: Scoping Review of Reviews Reporting on Digital Solutions Relevant for Older Adults

Silva¹,

Caravau²,

Martins³

et al. 2021

JMIR Hum Factors

View full text Add to dashboard Cite

Background The assessment of usability is a complex process that involves several steps and procedures. It is important to standardize the evaluation and reporting of usability procedures across studies to guide researchers, facilitate comparisons across studies, and promote high-quality usability studies. The first step to standardizing is to have an overview of how usability study procedures are reported across the literature. Objective This scoping review of reviews aims to synthesize the procedures reported for the different steps of the process of conducting a user-centered usability assessment of digital solutions relevant for older adults and to identify potential gaps in the present reporting of procedures. The secondary aim is to identify any principles or frameworks guiding this assessment in view of a standardized approach. Methods This is a scoping review of reviews. A 5-stage scoping review methodology was used to identify and describe relevant literature published between 2009 and 2020 as follows: identify the research question, identify relevant studies, select studies for review, chart data from selected literature, and summarize and report results. The research was conducted on 5 electronic databases: PubMed, ACM Digital Library, IEEE, Scopus, and Web of Science. Reviews that met the inclusion criteria (reporting on user-centered usability evaluation procedures for any digital solution that could be relevant for older adults and were published in English) were identified, and data were extracted for further analysis regarding study evaluators, study participants, methods and techniques, tasks, and test environment. Results A total of 3958 articles were identified. After a detailed screening, 20 reviews matched the eligibility criteria. The characteristics of the study evaluators and participants and task procedures were only briefly and differently reported. The methods and techniques used for the assessment of usability are the topics that were most commonly and comprehensively reported in the reviews, whereas the test environment was seldom and poorly characterized. Conclusions A lack of a detailed description of several steps of the process of assessing usability and no evidence on good practices of performing it suggests that there is a need for a consensus framework on the assessment of user-centered usability evaluation. Such a consensus would inform researchers and allow standardization of procedures, which are likely to result in improved study quality and reporting, increased sensitivity of the usability assessment, and improved comparability across studies and digital solutions. Our findings also highlight the need to investigate whether different ways of assessing usability are more sensitive than others. These findings need to be considered in light of review limitations.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Procedures of User-Centered Usability Assessment for Digital Solutions: Scoping Review of Reviews Reporting on Digital Solutions Relevant for Older Adults

Silva¹,

Caravau²,

Martins³

et al. 2021

JMIR Hum Factors

View full text Add to dashboard Cite

show abstract

“…Usability from the provider perspective is also crucial in health care app development. Recent studies have highlighted the importance of user experiences for both the patient and provider [ 18 ]. As such, we optimized the app for ease of use for the provider while also providing maximum flexibility to adapt to new procedures and surgeries.…”

Section: Discussionmentioning

confidence: 99%

A Smartphone App With a Digital Care Pathway for Patients Undergoing Spine Surgery: Development and Feasibility Study

Ponder¹,

Ansah-Yeboah²,

Charalambous³

et al. 2020

JMIR Perioper Med

View full text Add to dashboard Cite

Background There is a great unmet clinical need to provide patients undergoing spinal surgery and their caregivers with ongoing, high-quality care before and after surgery in an efficiency-focused health care environment. Objective The objective of this study is to design, develop, and evaluate the acceptability and feasibility of a novel planning-, outcomes-, and analytics-based smartphone app called ManageMySurgery (MMS) in patients undergoing elective spine surgery (MMS-Spine). Methods The development process of the MMS app was conducted over 2 sequential stages: (1) an evidence-based intervention design with refinement from surgeon and patient feedback and (2) feasibility testing in a clinical pilot study. We developed a novel, mobile-based, Health Insurance Portability and Accountability Act–compliant platform for interventional and surgical procedures. It is a patient-centric mobile health app that streamlines patients’ interactions with their care team. MMS divides the patient journey into phases, making it feasible to provide customized care pathways that meet patients’ unique needs. Patient-reported outcomes are easily collected and conform to the National Institutes of Health Patient-Reported Outcomes Measurement Information System (PROMIS) standard. Results We tested the feasibility of the MMS-Spine app with patients undergoing elective spine surgery at a large academic health system. A total of 47 patients undergoing elective spine surgery (26 cervical spine and 21 lumbar spine surgeries) downloaded and used MMS-Spine to navigate their surgical journey, quantify their baseline characteristics and postoperative outcomes, and provide feedback on the utility of the app in preparing for and recovering from their spinal surgery. The median age was 59.0 (range 33-77) years, 22 of the 47 patients (47%) were women, and 26 patients (55%) had commercial insurance. Of the 47 patients, a total of 33 (70%) logged in on an iOS device, 11 (23%) on an Android device, and 3 (6%) on a computer or tablet. A total of 17 of the 47 patients (36%) added a caregiver, of which 7 (41%) logged in. The median number of sign-ins was 2. A total of 38 of 47 patients (81%) completed their baseline preoperative PROMIS-29 outcomes, and 14 patients (30%) completed at least one PROMIS-29 survey during the postoperative period. Of the 24 patients who completed the MMS survey, 21 (88%) said it was helpful during preparation for their procedure, 16 (67%) said it was helpful during the postoperative period, and 23 (96%) said that they would recommend MMS to a friend or family member. Conclusions We used a patient-centered approach based on proven behavior change techniques to develop a comprehensive smartphone app for patients undergoing elective spine surgery. The optimized version of the app is ready for formal testing in a larger randomized clinical study to establish its cost-effectiveness and effect on patients’ self-management skills and long-term outcomes.

show abstract

“…Medline Complete, CINAHL Complete, PubMed, Embase, Scopus, Google, and Google Scholar were searched from January 2018 to April 2020, reflecting the period after the latest systematic reviews found from a preliminary search. 9 , 10 , 12 , 13 , 17 The reference lists of the systematic and narrative reviews identified from the systematic database search were screened. 6 , 9–13 , 17 , 18 No limitation was applied for the publication year for this backward searching.…”

Section: Methodsmentioning

confidence: 99%

“… 8 Several systematic reviews and narrative reviews have been published in recent years on methods or standards to evaluate health apps using various domains or criteria. 9–13 However, there has not been a deep investigation of the assessment criteria (ie, questions and statements used in frameworks) for domains and scoring mechanisms used, or of the validity and reliability of the assessment methods used by these evaluation frameworks. Previous reviews have illustrated many of the questions used in app evaluation frameworks but did not provide further analysis on the advantages and disadvantages or subjectivity and objectivity of questions in a way that would be useful for developing a general evaluation framework.…”

Section: Introductionmentioning

confidence: 99%

Scoping review: Development and assessment of evaluation frameworks of mobile health apps for recommendations to consumers

Hensher

Cooper

Dona

et al. 2021

Journal of the American Medical Informatics Association

View full text Add to dashboard Cite

Objective The study sought to review the different assessment items that have been used within existing health app evaluation frameworks aimed at individual, clinician, or organizational users, and to analyze the scoring and evaluation methods used in these frameworks. Materials and Methods We searched multiple bibliographic databases and conducted backward searches of reference lists, using search terms that were synonyms of “health apps,” “evaluation,” and “frameworks.” The review covered publications from 2011 to April 2020. Studies on health app evaluation frameworks and studies that elaborated on the scaling and scoring mechanisms applied in such frameworks were included. Results Ten common domains were identified across general health app evaluation frameworks. A list of 430 assessment criteria was compiled across 97 identified studies. The most frequently used scaling mechanism was a 5-point Likert scale. Most studies have adopted summary statistics to generate the total scoring of each app, and the most popular approach taken was the calculation of mean or average scores. Other frameworks did not use any scaling or scoring mechanism and adopted criteria-based, pictorial, or descriptive approaches, or “threshold” filter. Discussion There is wide variance in the approaches to evaluating health apps within published frameworks, and this variance leads to ongoing uncertainty in how to evaluate health apps. Conclusions A new evaluation framework is needed that can integrate the full range of evaluative criteria within one structure, and provide summative guidance on health app rating, to support individual app users, clinicians, and health organizations in choosing or recommending the best health app.

show abstract

Mobile health app usability and quality rating scales: a systematic review

Cited by 97 publications

References 98 publications

Procedures of User-Centered Usability Assessment for Digital Solutions: Scoping Review of Reviews Reporting on Digital Solutions Relevant for Older Adults

Procedures of User-Centered Usability Assessment for Digital Solutions: Scoping Review of Reviews Reporting on Digital Solutions Relevant for Older Adults

A Smartphone App With a Digital Care Pathway for Patients Undergoing Spine Surgery: Development and Feasibility Study

Scoping review: Development and assessment of evaluation frameworks of mobile health apps for recommendations to consumers

Contact Info

Product

Resources

About