Potential effectiveness and efficiency issues in usability evaluation within digital health: A systematic literature review

Maqbool, Bilal; Herold, Sebastian

doi:10.1016/j.jss.2023.111881

Cited by 20 publications

(9 citation statements)

References 110 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Of those that have, most have focused on digital health technologies such as electronic medical records systems [54] and mobile clinical decision support tools [55] that are not used for remote data capture. In 2023, Maqbool and Herold [8] published a systematic review of usability evaluations describing a broad suite of over 1000 digital health tools consisting mostly of mHealth applications and including a subset of 20 products approximately aligned to our definition of sDHT, including fitness/activity trackers, digital sphygmomanometers, and wearable fall risk assessment systems. Compared to our study, Maqbool and Herold found relatively increased rates of clinician and carepartner participation, and reporting of learnability, eficiency, and memorability.…”

Section: Discussionmentioning

confidence: 99%

“…Although regulatory guidance and a published framework focused on verification, analytical validation, and clinical validation processes for sDHTs have been widely adopted [1,[4][5][6], detailed best practices focused on human factors, human-centered design, and/or usability (defined in Box 1) of sDHTs have not been clearly described. Given that sDHTs (A) encompass a wide spectrum of tools which may or may not meet the United States Food and Drug Administration (FDA) definition of a medical device [4]; (B) take various forms such as wearable, ingestible, implantable, and ambient tools [2]; and (C) are applicable to both clinical research and clinical practice [7], the methodological approaches taken towards sDHT usability evaluation have varied markedly [8,9].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A systematic scoping review of studies describing human factors, human-centered design, and usability of sensor-based digital health technologies

Tandon,

Cobb,

Centra

et al. 2024

Preprint

View full text Add to dashboard Cite

BackgroundIncreasing adoption of sensor-based digital health technologies (sDHTs) in recent years has cast light on the many challenges in implementing these tools into clinical trials and patient care at scale across diverse patient populations; however, the methodological approaches taken towards sDHT usability evaluation have varied markedly.ObjectiveTo elucidate the current landscape of studies reporting data related to sDHT human factors, human-centered design, and/or usability.MethodsWe conducted a systematic scoping review of studies published between 2013 and 2023 and indexed in PubMed, in which data related to sDHT human factors, human-centered design, and/or usability were reported. Following a systematic screening process, we extracted the study design; participant sample; the sDHT(s) used; the method(s) of data capture; and the type(s) of usability-related data captured.ResultsOur literature search returned 442 papers, of which 85 were found to be eligible and 83 were available for data extraction and not under embargo. In total, 164 sDHTs were evaluated; 141 were wearable tools while the remaining 23 were ambient tools. The majority of studies (n=55; 66%) reported summative evaluations of final-design sDHTs. Almost all studies (n=82; 98%) captured data from targeted end-users, but only 18 (22%) captured data from additional users such as carepartners or clinicians. User satisfaction and ease of use were evaluated for >80% of sDHTs; however, learnability, eficiency, and memorability were reported for only 11 (13%), 4 (5%), and 2 sDHTs (2%), respectively. Fourteen sDHTs (17%) were evaluated according to the extent to which users were able to understand the clinical data or other information presented to them (understandability) and/or the actions or tasks they should complete in response (actionability). Notable gaps in reporting included the absence of a sample size rationale (reported for 25% of all studies and 31% of summative studies) and incomplete sociodemographic descriptive data (complete age, sex/gender, and race/ethnicity reported for 17% of studies).ConclusionsBased on our findings, we suggest four actionable recommendations for future studies that will help to advance the implementation of sDHTs: 1) Consider in-depth assessment of technology usability beyond user satisfaction and ease of use; 2) Expand recruitment to include important user groups such as clinicians and carepartners; 3) Report the rationale for key study design considerations including the sample size; and 4) Provide rich descriptive statistics regarding the study sample to allow a complete understanding of generalizability to other patient populations and contexts of use.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A systematic scoping review of studies describing human factors, human-centered design, and usability of sensor-based digital health technologies

Tandon,

Cobb,

Centra

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…VR-based rehabilitation by therapists is explained by eight attributes, namely learnability (easy to learn the system), appropriateness or usefulness (suitability of the product or system according to the patient's need), effectiveness (accuracy and completeness by patients in achieving specified goals), efficiency (efficient system use for high-level productivity), memorability (easy to remember system functions), patient error protection and recovery (degree of protection against errors by the system), satisfaction (the system should be pleasant and acceptable), and operability (ability to control and operate the system software) [16,17].…”

Section: Usability For Therapists: Enhancing Efficiency and Effective...mentioning

confidence: 99%

The Dual Importance of Virtual Reality Usability in Rehabilitation: A Focus on Therapists and Patients

Naqvi,

Mishra

et al. 2024

Cureus

View full text Add to dashboard Cite

Virtual reality (VR) has advanced in medical education and rehabilitation from basic graphical applications due to its ability to generate a virtual three-dimensional (3D) environment. This environment is mostly used to practice professional skills, plan surgery procedures, simulate surgeries, display 3D anatomy, and rehabilitate various disorders. VR has transformed the field of rehabilitation therapy by providing immersive and engaging experiences that go beyond traditional bounds, significantly improving patient care and therapeutic results. Considering the direct impact of VR on the efficacy of the treatment for both therapists and patients, its dual significance for usability and user experience cannot be overstated. The purpose of this article is to determine the synergistic association between VR accessibility and the rehabilitation process, highlighting the significance of VR technology in designing the future of rehabilitation therapy and demonstrating how advancing VR technology can improve therapeutic outcomes despite overcoming obstacles encountered during VR usage. In conclusion, VR offers a personalized, efficient, interesting, and engaging rehabilitative environment for patients, while also assisting therapists in cultivating empathy and efficiency and encouraging innovative approaches in treatment procedures.

show abstract

“…inquiry) and quantitative or so-called testing-based approaches presents several challenges [18]. The technology developers need to balance expert-driven insights from heuristic evaluations with user-centric data from tools like usability measurement estimations, requiring an intricate understanding of both approaches [19]. Qualitative methods, rich in contextual information, can be subjective.…”

Section: Of 24mentioning

confidence: 99%

Towards a Refined Heuristic Evaluation: Incorporating Hierarchical Analysis for Weighted Usability Assessment

Talero-Sarmiento,

Gonzalez-Capdevilla,

Granollers

et al. 2024

Preprint

View full text Add to dashboard Cite

This study delves into the realm of heuristic evaluation within usability testing, focusing on refining and applying a tailored algorithm that simplifies the implementation of the Analytic Hierarchy Process for heuristic prioritization. Addressing the prevalent challenge of disparate evaluation methodologies and the absence of a standardized process in usability testing, our research introduces a novel method that leverages the Analytic Hierarchy Process alongside a bespoke algorithm. This algorithm utilizes transitive properties for pairwise comparisons, significantly reducing the evaluative workload. This innovative approach facilitates the estimation of heuristic relevance irrespective of the number of items per heuristic or the item scale and streamlines the overall evaluation process. In addition to rigorous simulation testing of the tailored algorithm, we applied our method in a practical scenario, engaging seven usability experts in evaluating a web interface. This empirical application underscored our method’s capacity to diminish the requisite comparison count and spotlight areas of improvement for critically weighted yet underperforming heuristics. The outcomes of this study underscore the efficacy of our approach in enhancing the efficiency and organization of heuristic evaluations, paving the way for more structured and effective usability testing methodologies in future research endeavors.

show abstract

Potential effectiveness and efficiency issues in usability evaluation within digital health: A systematic literature review

Cited by 20 publications

References 110 publications

A systematic scoping review of studies describing human factors, human-centered design, and usability of sensor-based digital health technologies

A systematic scoping review of studies describing human factors, human-centered design, and usability of sensor-based digital health technologies

The Dual Importance of Virtual Reality Usability in Rehabilitation: A Focus on Therapists and Patients

Towards a Refined Heuristic Evaluation: Incorporating Hierarchical Analysis for Weighted Usability Assessment

Contact Info

Product

Resources

About