Scoping review: Development and assessment of evaluation frameworks of mobile health apps for recommendations to consumers

Hensher, Martin; Cooper, Paul; Dona, Sithara Wanni Arachchige; Angeles, Mary Rose; Nguyen, Dieu; Heynsbergh, Natalie; Chatterton, Mary Lou; Peeters, Anna

doi:10.1093/jamia/ocab041

Cited by 58 publications

(42 citation statements)

References 95 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is beyond the scope of this paper to define specific, measurable items that define whether a mental health app is accessible or not, however, the examples provided in Figure 1 outline some considerations that might be incorporated in such items. A recent synthesis of various evaluation frameworks identified 11 distinct questions related to accessibility covering areas of availability, offline modes, and vulnerable populations as the target audience ( 48 ). Another review has recommended the Matching Person to Technology (MPT) model as one framework to use when considering health apps for people with intellectual disabilities ( 30 ).…”

Section: Future Directions: Making Mental Health Apps More Accessiblementioning

confidence: 99%

Accessibility and Digital Mental Health: Considerations for More Accessible and Equitable Mental Health Apps

Bunyi

Ringland

Schueller

2021

Front. Digit. Health

View full text Add to dashboard Cite

Digital mental health is often touted as a solution to issues of access to mental health care. However, there has been little research done to understand the accessibility of digital mental health, especially for those with disabilities. In this piece, we define accessibility as it relates to mental health apps, describe the current state of accessibility in the digital world broadly and in mental health apps more specifically, outline why accessibility matters in mental health apps, and identify future steps to better incorporate accessibility into research and development of mental health apps.

show abstract

Section: Future Directions: Making Mental Health Apps More Accessiblementioning

confidence: 99%

Accessibility and Digital Mental Health: Considerations for More Accessible and Equitable Mental Health Apps

Bunyi

Ringland

Schueller

2021

Front. Digit. Health

View full text Add to dashboard Cite

show abstract

“…Several frameworks exist to assess the different functionalities of smartphone apps. Variance in these frameworks can lead to difficulties evaluating smartphone apps [ 40 ]. To ensure the safety and privacy of end users across trials, standardized approaches for the assessment and production of smartphone apps are required.…”

Section: Discussionmentioning

confidence: 99%

Designing Supportive e-Interventions for Partners of Men With Prostate Cancer Using Female Partners’ Experiences: Qualitative Exploration Study

Heynsbergh¹,

Gréen²,

Jongebloed³

et al. 2022

JMIR Cancer

View full text Add to dashboard Cite

Background Partners of men living with prostate cancer (PCa) can experience a variety of unmet needs that are largely unaddressed by health care professionals. There is limited evidence to suggest which approach may be most effective in supporting partners’ unmet needs and further research is required to determine how to provide support to caregivers and how technology solutions can be designed. Objective This study aims to explore the experience of partners of men living with PCa and their perceptions of the potential role of information technology in supporting their needs. Methods A qualitative descriptive methodology using focus groups and phone interviews was used. Purposive sampling was used to recruit people attending a national conference supported by a national PCa organization. Interview guides were adapted from an existing evidence-based smartphone app for caregivers of people with colorectal cancer. Sessions were audio recorded and transcribed verbatim. A coding framework was developed, and transcripts were coded line by line into the framework. Codes within the framework were grouped into descriptive categories that were then developed into analytical themes. Results A total of 17 female partners participated in the study, with an average age of 64 (SD 8.5) years. The following two main themes emerged: In the first theme, that is, How technology can be shaped to support female partners of prostate cancer survivors, the content and design of the smartphone app was discussed in addressing female partners’ needs. The following four subthemes were developed: getting support from social networks and resources, the lack of relevant information, demystifying future care expectations during and following a PCa diagnosis, and delivering the smartphone app—to whom and from whom. In the second theme, that is, The benefits and barriers of technology, the suitability of smartphone apps as a supportive modality for female partners was described. This included three subthemes: the smartphone app as an appropriate modality for supporting female partners, the future anticipated benefits of using the smartphone app, and concerns for storing and accessing information on the internet. Conclusions A smartphone app may be a suitable modality for providing information and peer support to female partners of men living with PCa. There is a need to provide peer support for female partners in future interventions to ensure that female partners’ intimacy and daily practical needs are met.

show abstract

“…Hensher et al studied evaluation frameworks for mobile health apps. 19 They identified 430 assessment criteria; 269 of them are objective while 161 are subjective criteria that rather focus on user perception or intuition. Frameworks that have been used for evaluating mobile health applications include Mobile App Rating Scale, 20 System Usability Scale, Health-ITUES, 21 or the Technology Acceptance Model.…”

Section: Related Workmentioning

confidence: 99%

Evaluation Metrics for Health Chatbots: A Delphi Study

et al. 2021

View full text Add to dashboard Cite

Background In recent years, an increasing number of health chatbots has been published in app stores and described in research literature. Given the sensitive data they are processing and the care settings for which they are developed, evaluation is essential to avoid harm to users. However, evaluations of those systems are reported inconsistently and without using a standardized set of evaluation metrics. Missing standards in health chatbot evaluation prevent comparisons of systems, and this may hamper acceptability since their reliability is unclear. Objectives The objective of this paper is to make an important step toward developing a health-specific chatbot evaluation framework by finding consensus on relevant metrics. Methods We used an adapted Delphi study design to verify and select potential metrics that we retrieved initially from a scoping review. We invited researchers, health professionals, and health informaticians to score each metric for inclusion in the final evaluation framework, over three survey rounds. We distinguished metrics scored relevant with high, moderate, and low consensus. The initial set of metrics comprised 26 metrics (categorized as global metrics, metrics related to response generation, response understanding and aesthetics). Results Twenty-eight experts joined the first round and 22 (75%) persisted to the third round. Twenty-four metrics achieved high consensus and three metrics achieved moderate consensus. The core set for our framework comprises mainly global metrics (e.g., ease of use, security content accuracy), metrics related to response generation (e.g., appropriateness of responses), and related to response understanding. Metrics on aesthetics (font type and size, color) are less well agreed upon—only moderate or low consensus was achieved for those metrics. Conclusion The results indicate that experts largely agree on metrics and that the consensus set is broad. This implies that health chatbot evaluation must be multifaceted to ensure acceptability.

show abstract

Scoping review: Development and assessment of evaluation frameworks of mobile health apps for recommendations to consumers

Cited by 58 publications

References 95 publications

Accessibility and Digital Mental Health: Considerations for More Accessible and Equitable Mental Health Apps

Accessibility and Digital Mental Health: Considerations for More Accessible and Equitable Mental Health Apps

Designing Supportive e-Interventions for Partners of Men With Prostate Cancer Using Female Partners’ Experiences: Qualitative Exploration Study

Evaluation Metrics for Health Chatbots: A Delphi Study

Contact Info

Product

Resources

About