Systematic review of applied usability metrics within usability evaluation methods for hospital electronic healthcare record systems

Abstract

Background and objectives: Electronic healthcare records have become central to patient care. Evaluation of new systems include a variety of usability evaluation methods or usability metrics (often referred to interchangeably as usability components or usability attributes). This study reviews the breadth of usability evaluation methods, metrics, and associated measurement techniques that have been reported to assess systems designed for hospital staff to assess inpatient clinical condition.Methods: Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology, we searched Medline, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, and Open Grey from 1986 to 2019. For included studies, we recorded usability evaluation methods or usability metrics as appropriate, and any measurement techniques applied to illustrate these. We classified and described all usability evaluation methods, usability metrics, and measurement techniques. Study quality was evaluated using a modified Downs and Black checklist. Results:The search identified 1336 studies. After abstract screening, 130 full texts were reviewed. In the 51 included studies 11 distinct usability evaluation methods were identified. Within these usability evaluation methods, seven usability metrics were reported.The most common metrics were ISO9241-11 and Nielsen's components. An additional "usefulness" metric was reported in almost 40% of included studies. We identified 70 measurement techniques used to evaluate systems. Overall study quality was reflected in a mean modified Downs and Black checklist score of 6.8/10 (range 1-9) 33% studies classified as "high-quality" (scoring eight or higher), 51% studies "moderate-quality" (scoring 6-7), and the remaining 16% (scoring below five) were "low-quality." Conclusion:There is little consistency within the field of electronic health record systems evaluation. This review highlights the variability within usability methods, metrics,

Artificial Intelligence, Healthcare, Clinical Genomics, and Pharmacogenomics Approaches in Precision Medicine

“…Academic healthcare systems are research based and can be characterized by limited clinical data management and periodic data flow with the overall goal of improving patient treatment ( Rim et al, 2016 ; Kim et al, 2017 ; Ahmed et al, 2019a ). An electronic healthcare record (EHR) is the collection of patient health information that is digitally stored ( Knaup et al, 2007 ; Wronikowska et al, 2021 ). The compilation of patient information serves to also create an overall understanding of a given population’s health status, for example, the frequency of specific diseases within the population or a specific sub-group ( Knaup et al, 2007 ; Yeh et al, 2020 ).…”

“…EHRs can assist healthcare providers in efficiently diagnosing specific rare medical cases that may not be encountered often ( Knaup et al, 2007 ; Yeh et al, 2020 ; Wronikowska et al, 2021 ). In recent years, it has become more difficult to improve the health outcomes for a patient by a drastic margin without raising their out-of-pocket costs ( Al-Hanawi et al, 2021 ), thus making it difficult for timely progress toward better patient care ( Pastorino et al, 2019 ; Al-Hanawi et al, 2021 ).…”

Non-empirical ISO 9241-210:2019-based usability evaluation using fuzzy inference analyser: a special issue on interactive social robots

“…Common methods used for expert-based usability assessment are varied, such as reviewing guides based on interaction design guidelines, innovative assessments, reviewing cognitive enhancements, reviewing usage paths, formal usability inspections and innovative marches (Nielsen, 1994). In recent years, several review studies have been conducted on usability evaluation methods (Davis et al, 2020;Li et al, 2019;Rauf et al, 2019;Vargas Caceres et al, 2018;Wronikowska et al, 2021). Recently, some of these expert-based techniques have proposed new models of evaluation factors (Lee and Lee, 2019;Veral and Macias, 2019), some extended methods include new heuristics (Othman et al, 2018;Yanez-Gomez et al, 2019).…”

Systematic Review of the Usability Evaluation Methodology of Health Information Systems

“…Terdapat beberapa metode evaluasi usability untuk menilai sistem informasi. Metode evaluasi usability yang didapatkan dari literatur termasuk uji coba pengguna, kuesioner, wawancara, evaluasi heuristik dan cognitive walkthrough (Wronikowska et al, 2021). Tidak terdapat kriteria standar dalam membandingkan hasil dari setiap metode yang digunakan (Hartson et al, 2003).…”