Background Electronic health (eHealth) usability evaluations of rapidly developed eHealth systems are difficult to accomplish because traditional usability evaluation methods require substantial time in preparation and implementation. This illustrates the growing need for fast, flexible, and cost-effective methods to evaluate the usability of eHealth systems. To address this demand, the present study systematically identified and expert-validated rapidly deployable eHealth usability evaluation methods. Objective Identification and prioritization of eHealth usability evaluation methods suitable for agile, easily applicable, and useful eHealth usability evaluations. Methods The study design comprised a systematic iterative approach in which expert knowledge was contrasted with findings from literature. Forty-three eHealth usability evaluation methods were systematically identified and assessed regarding their ease of applicability and usefulness through semi-structured expert interviews with 10 European usability experts and systematic literature research. The most appropriate eHealth usability evaluation methods were selected stepwise based on the experts' judgements of their ease of applicability and usefulness. Results Of these 43 eHealth usability evaluation methods identified as suitable for agile, easily applicable, and useful eHealth usability evaluations, 10 were recommended by the experts based on their usefulness for rapid eHealth usability evaluations. The three most frequently recommended eHealth usability evaluation methods were Remote User Testing, Expert Review, and Rapid Iterative Test and Evaluation Method. Eleven usability evaluation methods, such as Retrospective Testing, were not recommended for use in rapid eHealth usability evaluations. Conclusion We conducted a systematic review and expert-validation to identify rapidly deployable eHealth usability evaluation methods. The comprehensive and evidence-based prioritization of eHealth usability evaluation methods supports faster usability evaluations, and so contributes to the ease-of-use of emerging eHealth systems.

Previous research has shown that there are a variety of eHealth usability evaluation methods suitable for agile, easily applicable, and useful eHealth usability evaluations. However, it is unclear whether such eHealth usability evaluation methods are also applicable with elderly users. This study aims to examine the challenges in applying eHealth usability evaluation methods with elderly users and how these challenges can be overcome. We chose three established eHealth usability evaluation methods to evaluate an eHealth intervention: (1) Co-Discovery Evaluation, (2) Cooperative Usability Testing, and (3) Remote User Testing combined with Think Aloud. The case study was conducted with seven Austrian elderly users. We supplemented the case study (March, 2021) with a systematic review (March, 2022) to identify (1) applied eHealth usability evaluation methods to elderly and (2) challenges of eHealth usability evaluations with elderly. Our results showed that Remote User Testing combined with Think Aloud could successfully be applied to evaluate the eHealth intervention with elderly users. However, Cooperative Usability Testing and Co-Discovery Evaluation were not suitable. The results of the systematic review showed that user-based eHealth usability evaluation methods are mostly applied to conduct eHealth usability evaluations with elderly users. Overall, the results showed that not all established eHealth usability evaluation methods are applicable with elderly users. Based on the case study and the systematic review, we developed 24 recommendations on how to deal with challenges during eHealth usability evaluations. The recommendations contribute to improving the accessibility, acceptability, and usability of eHealth interventions by the elderly.